The Bennett-Orlicz Norm

Abstract

van de Geer and Lederer (Probab. Theory Related Fields 157(1-2), 225–250, ²⁰¹³) introduced a new Orlicz norm, the Bernstein-Orlicz norm, which is connected to Bernstein type inequalities. Here we introduce another Orlicz norm, the Bennett-Orlicz norm, which is connected to Bennett type inequalities. The new Bennett-Orlicz norm yields inequalities for expectations of maxima which are potentially somewhat tighter than those resulting from the Bernstein-Orlicz norm when they are both applicable. We discuss cross connections between these norms, exponential inequalities of the Bernstein, Bennett, and Prokhorov types, and make comparisons with results of Talagrand (Ann. Probab., 17(4), 1546–1570, ^{1989, 1991}), and Boucheron et al. (2013).

1 Orlicz Norms and Maximal Inequalities

Let Ψ be an increasing convex function from [0, ∞) onto [0, ∞). Such a function is called a Young-Orlicz modulus by Dudley (1999), and a Young modulus by de la Peña and Giné (1999). Let X be a random variable. The Orlicz norm ‖X‖_Ψ is defined by

$$\Vert X{\Vert }_{\mathrm{\Psi }}=\inf \left\{c>0:E\mathrm{\Psi }\left(\frac{|X|}{c}\right)\le 1\right\},$$

where the infimum over the empty set is ∞. By Jensen’s inequality it is easily shown that this does define a norm on the set of random variables for which ‖X‖_Ψ is finite. The most important functions Ψ for a variety of applications are those of the form Ψ(x) = exp(x^p)−1 ≡ Ψ_p(x) for p ≥ 1, and in particular Ψ₁ and Ψ₂ corresponding to random variables which are “sub-exponential” or “sub-Gaussian” respectively. See Krasnoseľskiĭ and Rutickiĭ (1961), Dudley (1999), Arcones and Giné (1995), de la Peña and Giné (1999), & van der Vaart and Wellner (1996) for further background on Orlicz norms, and see Rao and Ren (1991), Krasnoseľskiĭ and Rutickiĭ (1961), & Hewitt and Stromberg (1975) for more information about Birnbaum-Orlicz spaces.

The following useful lemmas are from van der Vaart and Wellner (1996), pages 95-97, and Arcones and Giné (1995) (see also de la Peña and Giné (1999), pages 188-190), respectively.

Lemma 1.1

Let Ψ be a convex, nondecreasing, nonzero function with Ψ(0) = 0 and lim sup_x,y→∞ Ψ(x)Ψ(y)/Ψ(cxy) < ∞ for some constant c. Then, for any random variables X₁, …, X_m,

$$\Vert \underset{1\le j\le m}{\max }{X}_i{\Vert }_{\mathrm{\Psi }}\le K{\mathrm{\Psi }}^{-1}(m)\underset{1\le j\le m}{\max }\Vert X_i{\Vert }_{\mathrm{\Psi }}$$ (1.1)

where K is a constant depending only on Ψ.

Lemma 1.2

Let Ψ be a Young Modulus satisfying

$$\underset{x,y\to \mathrm{\infty }}{\lim \sup }\frac{{\mathrm{\Psi }}^{-1}(xy)}{{\mathrm{\Psi }}^{-1}(x){\mathrm{\Psi }}^{-1}(y)}<\mathrm{\infty }\mathit{and}\underset{x\to \mathrm{\infty }}{\lim \sup }\frac{{\mathrm{\Psi }}^{-1}(x^2)}{{\mathrm{\Psi }}^{-1}(x)}<\mathrm{\infty }.$$

Then for some constant M depending only on Ψ and every sequence of random variables {X_k: k ≥ 1},

$${\Vert \underset{k\ge 1}{\sup }\frac{|X_k|}{{\mathrm{\Psi }}^{-1}(k)}\Vert }_{\mathrm{\Psi }}\le M\underset{k\ge 1}{\sup }\Vert X_k{\Vert }_{\mathrm{\Psi }}.$$ (1.2)

The inequality (1.1) shows that if Orlicz norms for individual random variables ${\left\{X_i\right\}}_{i=1}^m$ are under control, then the Ψ–Orlicz norm of the maximum of the X_i’s is controlled by a constant times Ψ⁻¹(m) times the maximum of the individual Orlicz norms. The inequality (1.2) shows a stronger related Orlicz norm control of the supremum of an entire sequence X_k divided by Ψ⁻¹(k) if the supremum of the individual Orlicz norms is finite. Lemma 1.2 implies Lemma 1.1 for Young functions of exponential type (such as Ψ_p(x) = exp(x^p) − 1 with p ≥ 1), but it does not hold for power type Young functions such as Ψ(x) = x^p, p ≥ 1. These latter Young functions continue to be covered by Lemma 1.1. Arcones and Giné (1995) carefully define Young moduli Ψ_p(x) = exp(x^p) − 1 for all p > 0 and use Lemma 1.2 to establish laws of the iterated logarithm for U-statistics.

A general theme is that if Ψ_a ≤ Ψ_b and we have control of the individual Ψ_b Orlicz norms, then Lemma 1.1 or Lemma 1.2 applied with Ψ = Ψ_b will yield a better bound than with Ψ = Ψ_a in the sense that ${\mathrm{\Psi }}_b^{-1}(m)\le {\mathrm{\Psi }}_a^{-1}(m)$.

Here we are interested in functions Ψ of the form

$$\mathrm{\Psi }(x)=\exp (h(x))-1$$ (1.3)

where h is a nondecreasing convex function with h(0) = 0 not of the form x^p. In fact, the particular functions h of interest here are (scaled versions of):

$$h_0(x)=\frac{x^2}{2(1+x)},$$

$$h_1(x)=1+x-\sqrt{1+2x},$$

$$h_2(x)=h(1+x)=(1+x)\log (1+x)-x,$$

$$h_4(x)=(x/2)\text{arcsinh}(x/2),$$

$$h_5(x)=(x)\text{arcsinh}(x/2)-2\left(\cosh (\text{arcsinh}(x/2))-1\right)$$

for the particular h(x) ≡ x(log x − 1) + 1. The functions h₀ and h₁ are related to Bernstein exponential bounds and refinements thereof due to Birgé and Massart (1998), while the function h₂ is related to Bennett’s inequality (Bennett, 1962), and h₄ is related to Prokhorov’s inequality (Prokhorov, 1959).

van de Geer and Lederer (2013) studied the family of Orlicz norms defined in terms of scaled versions of h₁, and called called them Bernstein-Orlicz norms. Our primary goal here is to compare and contrast the Orlicz norms defined in terms of h₀, h₁, h₂, and h₄. We begin in the next section by reviewing the Bernstein-Orlicz norm(s) as defined by van de Geer and Lederer (2013). Section 3 gives corresponding results for what we call the Bennett-Orlicz norm(s) corresponding to the function h₂. In Section 4 we give further comparisons and two applications.

2 The Bernstein-Orlicz Norm

For a given number L > 0, van de Geer and Lederer (2013) have defined the Bernstein-Orlicz norm $\Vert X{\Vert }_{{\mathrm{\Psi }}_L}$ with

$${\mathrm{\Psi }}_L(x)\equiv {\mathrm{\Psi }}_1(x;L)\equiv \exp \left\{{\left(\frac{\sqrt{1+2Lx}-1}{L}\right)}^2\right\}-1=\exp \left\{\frac{2}{L^2}{h}_1(Lx)\right\}-1.$$ (2.1)

It is easily seen that

$${\mathrm{\Psi }}_1(x;L)~\{\begin{array}{ll}\exp (x^2)-1\hfill & \text{for }Lx\text{small},\hfill \\ \exp (2x/L)-1\hfill & \text{for }Lx\text{large}.\hfill \end{array}$$

The following three lemmas of van de Geer and Lederer (2013) should be compared with the development on page 96 of van der Vaart and Wellner (1996).

Lemma 2.1

Let $\tau \equiv \Vert Z{\Vert }_{{\mathrm{\Psi }}_1(·;L)}$. Then

$$P(|Z|>\tau [\sqrt{t}+2^{-1}Lt])\le 2e^{-t}\mathit{\text{for all}}t>0;$$

or, equivalently, with $h_1^{-1}(y)\equiv y+\sqrt{2y}$,

$$P\left(|Z|>(\tau /L)h_1^{-1}\left(\frac{L^{2}t}{2}\right)\right)\le 2e^{-t}\mathit{\text{for all}}t>0;$$

or

$$P(|Z|>z)\le 2\exp \left(-\frac{2}{L^2}{h}_1\left(\frac{Lz}{\tau }\right)\right)\mathit{\text{for all}}z>0.$$ (2.2)

Lemma 2.2

Suppose that for some τ and L > 0 we have

$$P(|Z|\ge \tau [\sqrt{t}+2^{-1}Lt])\le 2e^{-t}\mathit{\text{for all}}t>0.$$

Equivalently, the inequality (2.2) holds. Then $\Vert Z{\Vert }_{{\mathrm{\Psi }}_1(·;\sqrt{3}L)}\le \sqrt{3}\tau$.

Example 2.1

Suppose that X ~ Poisson(ν). Then it is well-known (see e.g. Boucheron et al. (2013), page 23), that

$$P(|X-\nu |\ge z)\le 2\exp (-\nu h_2(z/\nu ))\le 2\exp (-9\nu h_1(z/(3\nu )))$$

where h₂(x) = h(1 + x) = (x + 1) log(x + 1) − x. Thus the inequality involving h₁ holds with 9ν = 2/L² and 1/(3ν) = L/τ. Thus $L=\sqrt{2/(9\nu )}=3^{-1}\sqrt{2/\nu }$, $\tau =L3\nu =\sqrt{2\nu }$. We conclude from Lemma 2.2 that

$$\Vert X-\nu {\Vert }_{{\mathrm{\Psi }}_1(·;\sqrt{2/3\nu })}\le \sqrt{6\nu }.$$

Pisier (1983) and Pollard (1990) showed how to bound the Orlicz norm of the maximum of random variables with bounded Orlicz norms; see also de la Peña and Giné (1999), section 4.3, and van der Vaart and Wellner (1996), Lemma 2.2.2, page 96. The following bound for the expectation of the maximum was given by van de Geer and Lederer (2013); also see Boucheron et al. (2013), Theorem 2.5, pages 32-33.

Lemma 2.3

Let τ and L be positive constants, and let Z₁, …, Z_m be random variables satisfying ${\max }_{1\le j\le m}\Vert Z_j{\Vert }_{{\mathrm{\Psi }}_L}\le \tau$. Then

$$E\{\underset{1\le j\le m}{\max }|Z_j|\}\le \tau {\mathrm{\Psi }}_1^{-1}(m;L)=\tau \left\{\sqrt{\log (1+m)}+\frac{L}{2}\log (1+m)\right\}.$$ (2.3)

Corollary 2.1

For m ≥ 2

$$E\{\underset{1\le j\le m}{\max }|Z_j|\}\le \tau {\mathrm{\Psi }}_1^{-1}(m;L)\le 2\max \{\tau ,L\tau /2\}\log (1+m).$$

In particular when Z_j ~ Poisson(ν) for 1 ≤ j ≤ m

$$E\{\underset{1\le j\le m}{\max }|Z_j|\}\le \tau {\mathrm{\Psi }}_1^{-1}(m;L)\le 2\{\sqrt{2\nu }\vee 1/3\}\log (1+m).$$

Proof

This follows from Lemma 2.3 since $\sqrt{x}\le x$ for x ≥ 1. The Poisson(ν) special case then follows from Example 2.1.

It will be helpful to relate Ψ₁(·; L) to several functions appearing frequently in the theory of exponential bounds as follows: for x ≥ 0, we define

$$h(x)=x(\log x-1)+1,$$

$$h_1(x)=1+x-\sqrt{1+2x},$$ (2.4)

$$h_0(x)=\frac{x^2}{2(1+x)}.$$

It is easily shown (see e.g. Boucheron et al. (2013) Exercise 2.8, page 47) that

$$9h_0(x/3)\le 9h_1(x/3)\le h(1+x).$$ (2.5)

A trivial restatement of the inequality on the left above and some algebra and easy inequalities yield

$$h_0(x)\le h_1(x)\le 2h_0(x)\le h_0(2x).$$ (2.6)

The latter inequalities imply that the Orlicz norms based on h₀ and h₁ are equivalent up to constants.

One reason the functions h₀ and h₁ are so useful is that they both have explicit inverses: from Boucheron, Lugosi, and Massart (2013), page 29, for h₁ and direct calculation for h₀,

$$h_1^{-1}(y)=y+\sqrt{2y},\text{for }y\ge 0,$$

$$h_0^{-1}(y)=y+\sqrt{y^2+2y}.$$

To relate the inequalities in Lemmas 2.1 and 2.2 to more standard inequalities (with names) we note that

$$\sqrt{t}+2^{-1}Lt=\frac{1}{L}{h}_1^{-1}\left(\frac{L^{2}t}{2}\right).$$

This implies immediately that the inequality in Lemma 2.2 can be rewritten as

$$P(|Z|>z)\le 2\exp \left(-\frac{2}{L^2}{h}_1\left(\frac{Lz}{\tau }\right)\right)\le 2\exp \left(-\frac{2}{L^2}{h}_0\left(\frac{Lz}{\tau }\right)\right)=2\exp \left(-\frac{z^2}{{\tau }^2+L\tau z}\right)\text{for all }z>0.$$

Here is a formal statement of a proposition relating exponential tail bounds in the traditional Bernstein form in terms of h₀ to tail bounds in terms of the (larger) function h₁.

Proposition 2.1

Suppose that a random variable Z satisfies

$$P(|Z|>z)\le 2\exp \left(-\frac{z^2}{2(A+Bz)}\right)=2\exp \left(-\frac{A}{B^2}{h}_0\left(\frac{Bz}{A}\right)\right)\mathit{\text{for all}}z>0$$ (2.7)

for numbers A, B > 0. Then the hypothesis of Lemma 2.2 holds with L and τ given by L² = 2B²/A and τ = 2^3/2A^1/2:

$$P(|Z|>z)\le 2\exp \left(-\frac{A}{B^2}{h}_1\left(\frac{Bz}{2A}\right)\right)$$ (2.8)

$$=2\exp \left(-\frac{2}{L^2}{h}_1\left(\frac{Lz}{\tau }\right)\right)\mathit{\text{for all}}z>0.$$ (2.9)

Proof

This follows from Eq. 2.6 and elementary manipulations.

The classical route to proving inequalities of the form given in Eq. 2.7 for sums of independent random variables is via Bernstein’s inequality; see for example (van der Vaart and Wellner, 1996) Lemmas 2.2.9 and 2.2.11, pages 102 and 103, or Boucheron et al. (2013), Theorem 2.10, page 37. But the recent developments of concentration inequalities via Stein’s method yields inequalities of the form given in Eq. 2.7 for many random variables Z which are not sums of independent random variables: see, for example, Ghosh and Goldstein (2011a), Ghosh and Goldstein (2011b), & Goldstein and Iṡlak (2014). The point of the previous proposition is that (up to constants) these inequalities in terms of h₀ can be re-expressed in terms of the (larger) function h₁.

3 Bennett’s Inequality and the Bennett-Orlicz Norm

We begin with a statement of a version of Bennett’s inequality for sums of bounded random variables; see Bennett (1962), Shorack and Wellner (1986), & Boucheron et al. (2013). Let h(x) ≡ x(log x−1)+1 and h₂(x) ≡ h(1+x). This function arises in Bennett’s inequality for bounded random variables and elsewhere; see e.g. Bennett (1962), Shorack and Wellner (1986), & Boucheron et al. (2013), page 35 (but note that their h is our h₂ = h(1 + ·)). As noted in Example 1 above, the function h also appears in exponential bounds for Poisson random variables: see Shorack and Wellner (1986) page 485, and Boucheron et al. (2013) page 23.

Proposition 3.1

(Bennett) (i) Let X₁, …, X_n be independent with max_1≤j≤n(X_j − μ_j) ≤ b, E(X_j) = μ_j, $\mathit{\text{Var}}(X_j)={\sigma }_{n,j}^2$. Let $\mu \equiv {\displaystyle {\sum }_{j=1}^n{\mu }_j/n}$, ${\sigma }_n^2\equiv ({\sigma }_{n,1}^2+\cdots +{\sigma }_{n,n}^2)/n$. Then with ψ(x) ≡ 2h(1 + x)/x²,

$$P(\sqrt{n}({\overline{X}}_n-\mu )\ge z)\le \exp \left(-\frac{z^2}{2{\sigma }_n^2}\psi \left(\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right)=\exp \left(-\frac{n{\sigma }_n^2}{b^2}h\left(1+\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right)=\exp \left(-\frac{n{\sigma }_n^2}{b^2}{h}_2\left(\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right)$$ (3.1)

for all z > 0.

(ii) If, in addition, max_1≤j≤n |X_j − μ_j| ≤ b, then

$$P(|\sqrt{n}({\overline{X}}_n-\mu )|\ge z)\le 2\exp \left(-\frac{z^2}{2{\sigma }_n^2}\psi \left(\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right)=2\exp \left(-\frac{n{\sigma }_n^2}{b^2}h\left(1+\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right)=2\exp \left(-\frac{n{\sigma }_n^2}{b^2}{h}_2\left(\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right).$$

Using the inequality h(1 + x) ≥ 9h₁(x/3), it follows that

$$P(|\sqrt{n}({\overline{X}}_n-\mu )|\ge z)\le 2\exp \left(-\frac{9n{\sigma }_n^2}{b^2}{h}_1\left(\frac{zb}{3\sqrt{n}{\sigma }_n^2}\right)\right).$$

Thus an inequality of the form of that in Lemma 2.1 holds with $2/L^2=9n{\sigma }_n^2/b^2$ and $L/\tau =b/(3\sqrt{n}{\sigma }_n^2)$. Thus $L=\sqrt{2/9b}/(\sqrt{n}{\sigma }_n)$ and $\tau =L3\sqrt{n}{\sigma }_n^2/b=\sqrt{2}{\sigma }_n$. It follows from Lemma 2.2 that

$$\Vert \sqrt{n}({\overline{X}}_n-\mu ){\Vert }_{{\mathrm{\Psi }}_1(·;\sqrt{3}L)}\le \sqrt{6}{\sigma }_n,$$

or

$$\Vert \sqrt{n}({\overline{X}}_n-\mu ){\Vert }_{{\mathrm{\Psi }}_1(·;\sqrt{2/3}b/(\sqrt{n}{\sigma }_n))}\le \sqrt{6}{\sigma }_n.$$

But this bound has not taken advantage of the fact the the first bound above involves the function h (or h₂) rather than h₁. It would seem to be of potential interest to develop an Orlicz norm based on the function h₂ ≡ h(1 + ·) rather than the function h₁. Motivated by the first inequality in Proposition 3.1, we define for each L > 0 a new Orlicz norm based on the function h₂ as follows.

$${\mathrm{\Psi }}_2(x;L)\equiv \exp \left(\frac{2}{L^2}{h}_2(Lx)\right)-1.$$

Since h₂ is convex, h₂(0) = 0, and h₂ is increasing on [0, ∞), it follows that Ψ₂(·; L) defines a valid Orlicz norm (as defined in Section 1) for each L:

$$\Vert X{\Vert }_{{\mathrm{\Psi }}_2(·;L)}=\inf \left\{c>0:E{\mathrm{\Psi }}_2\left(\frac{|X|}{c};L\right)\le 1\right\};$$ (3.2)

We call $\Vert X{\Vert }_{{\mathrm{\Psi }}_2(·;L)}$ the Bennett-Orlicz norm of X. Note that with ψ(Lx) ≡ x⁻²(2/L²)h₂(Lx),

$${\mathrm{\Psi }}_2(x;L)=\exp \left(x^2\psi (Lx)\right)-1~\{\begin{array}{ll}\exp (x^2)-1\hfill & \text{for }Lx\text{small},\hfill \\ \exp \left(\frac{2x}{L}\log (Lx)\right)-1\hfill & \text{for }Lx\text{large}.\hfill \end{array}$$

We first relate Ψ₂(·; L) to Ψ₁(x; L) and to the usual Gaussian Orlicz norm defined by Ψ₂(x) = exp(x²) − 1.

Proposition 3.2

1. Ψ₂(x; L) ≤ exp(x²) − 1 = Ψ₂(x) for all x ≥ 0.

2. Ψ₂ (x; L) ≥ Ψ₁(x; L/3) for x ≥ 0.

Proof

(i) follows since ψ(x) ≡ 2x⁻²h(1 + x) ≤ 1 for all x ≥ 0; see Shorack and Wellner (1986), Proposition 11.1.1, page 441. To show that (ii) holds, note that by Eq. 2.1

$${\mathrm{\Psi }}_1(x;L/3)=\exp \left(\frac{2·9}{L^2}{h}_1(Lx/3)\right)-1.$$

Thus the claimed inequality in (ii) is equivalent to

$$\frac{2}{L^2}h(1+Lx)\ge \frac{2·9}{L^2}{h}_1(Lx/3),$$

or equivalently

$$h(1+Lx)\ge 9h_1(Lx/3).$$

But the inequality in the last display holds in view of Eq. 2.5.

Note that while h₁ and Ψ₁(·; L) have explicit inverses given in terms of $\sqrt{v}$ and log(1 + v) by Eqs. 2.7 and 2.3, inverses of the functions h₂ and Ψ₂(·; L) can only be written in terms of Lambert’s function (also called the product log function) W satisfying W (z) exp(W (z)) = z; see Corless et al. (1996). But this slight difficulty is easily overcome by way of several nice inequalities for W. By use of W and the inequalities developed in the Appendix, Section 6, we obtain the following proposition concerning ${\mathrm{\Psi }}_2^{-1}(·;L)$.

Proposition 3.3

1. ${\mathrm{\Psi }}_2^{-1}(y;L)\le {\mathrm{\Psi }}_1^{-1}(y;L/3)=\sqrt{\log (1+y)}+(L/6)\log (1+y)$ for y ≥ 0.

2. Furthermore, with W denoting the Lambert W function,

   $${\mathrm{\Psi }}_2^{-1}(y;L)=\frac{1}{L}{h}_2^{-1}\left(\frac{L^2}{2}\log (1+y)\right)=\frac{1}{L}\left\{\frac{\left((L^2/2)\log (1+y)-1\right)}{W\left(\left((L^2/2)\log (1+y)-1\right)/e\right)}-1\right\}.$$
3. If (L²/2) log(1 + y) ≥ 1, then

   $${\mathrm{\Psi }}_2^{-1}(y;L)\le L\frac{\log (1+y)}{\log ((L^2/2)\log (1+y)-1)}.$$
4. If (L²/2) log(1 + y) ≥ 5, then

   $${\mathrm{\Psi }}_2^{-1}(y;L)\le 2L\frac{\log (1+y)}{\log ((L^2/2)\log (1+y))}\le 2L\frac{\log (1+y)}{\log \log (1+y)}if\mathit{\text{also}}{L}^2/2\ge 1.$$
5. If (L²/2) log(1 + y) ≤ 9/4, then

   $${\mathrm{\Psi }}_2^{-1}(y;L)\le \sqrt{2\log (1+y)}.$$

6. ${\mathrm{\Psi }}_2^{-1}(y;L)\le \{\begin{array}{ll}(2.2/\sqrt{2})\sqrt{\log (1+y)}\hfill & if(L^2/2)\log (1+y)\le 1+e,\hfill \\ L\frac{\log (1+y)-1}{\log ((L^2/2)\log (1+y)-1)}-1\hfill & if(L^2/2)\log (1+y)>1+e.\hfill \end{array}$

Proof

(i) follows immediately from Proposition 3.2. (ii) follows from the definition of Ψ₂(·; L) and direct computation for the first part; the second part follows from Lemma 6.1. The inequality in (iii) follows from (ii) and Lemma 6.2. The first inequality in (iv) follows from (iii) since log(y − 1) ≥ (1/2) log y for y ≥ 4. The second inequality in (iv) follows by noting that

$$\log ((L^2/2)\log (1+y))=\log (L^2/2)+\log \log (1+y)\ge \log \log (1+y)$$

if L²/2 ≥ 1. (v) follows from (ii) and Lemma 6.3, part (iv).

Lemmas 2.1 and 2.2 by van de Geer and Lederer (2013) as stated in Section 2 should be compared with the development on page 96 of van der Vaart and Wellner (1996). We now show that the following analogues of Lemmas 2.1–2.3 hold for $\Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;L)}$.

Lemma 3.1

Let $\tau \equiv \Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;L)}$. Then

$$P\left(|Z|>\frac{\tau }{L}{h}_2^{-1}(L^{2}t/2)\right)\le 2e^{-t}\mathit{\text{for all}}t>0$$

where h₂(x) ≡ h(1 + x) and $h_2^{-1}$ is the inverse of h₂ (so that $h_2^{-1}(y)=h^{-1}(y)-1$)

Proof

Let y > 0. Since Ψ₂(x;L) = exp((2/L²)h₂(Lx)) − 1 = e^t − 1 implies h₂(Lx) = L²t/2, it follows that for any $c>\Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;L)}$ we have

$$P\left(\frac{|Z|}{c}>\frac{1}{L}{h}_2^{-1}(L^{2}t/2)\right)=P\left(h_2\left(\frac{L|Z|}{c}\right)>L^{2}t/2\right)=P\left(\frac{2}{L^2}{h}_2\left(\frac{L|Z|}{c}\right)>t\right)=P\left(\exp \left(\frac{2}{L^2}{h}_2\left(\frac{L|Z|}{c}\right)\right)-1>e^t-1\right)=P\left({\mathrm{\Psi }}_2\left(\frac{|Z|}{c};L\right)>e^t-1\right)\le \frac{E\left\{{\mathrm{\Psi }}_2\left(\frac{|Z|}{c};L\right)+1\right\}}{e^t}\to 2e^{-t}\text{as }c\downarrow \tau \equiv \Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;L)}.$$

Lemma 3.2

Suppose that for some τ > 0 we have

$$P\left(|Z|>\frac{\tau }{L}{h}_2^{-1}(L^{2}t/2)\right)\le 2e^{-t}\mathit{\text{for all}}t>0.$$

Equivalently,

$$P(|Z|>z)\le 2\exp \left(-\frac{2}{L^2}{h}_2\left(\frac{Lz}{\tau }\right)\right)=2\exp \left(-\frac{2}{L^2}h\left(1+\frac{Lz}{\tau }\right)\right)=2\exp \left(-\frac{z^2}{{\tau }^2}\psi \left(\frac{Lz}{\tau }\right)\right)\mathit{\text{for all}}z>0.$$

Then $\Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;\sqrt{3}L)}\le \sqrt{3}\tau$.

Proof

Let α, β > 0. We compute

$$E{\mathrm{\Psi }}_2\left(\frac{|Z|}{\alpha \tau };\beta L\right)={\displaystyle {\int }_0^{\mathrm{\infty }}P\left({\mathrm{\Psi }}_2\left(\frac{|Z|}{\alpha \tau };\beta L\right)\ge v\right)dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(\frac{2}{{\beta }^2{L}^2}{h}_2\left(\frac{\beta L|Z|}{\alpha \tau }\right)\ge \log (1+v)\right)}dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(\frac{\beta L|Z|}{\alpha \tau }\ge \tau h_2^{-1}\left(\frac{{\beta }^2{L}^2}{2}\log (1+v)\right)\right)}}dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(|Z|\ge \frac{\alpha \tau }{\beta L}{h}_2^{-1}\left(\frac{{\beta }^2{L}^2}{2}\log (1+v)\right)\right)}dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(|Z|\ge \frac{\alpha \tau }{\beta L}{h}_2^{-1}\left(\frac{{\beta }^2{L}^2}{2}t\right)\right)}{e}^{t}dt.$$

Choosing $\alpha =\beta =\sqrt{3}$ this yields

$$E{\mathrm{\Psi }}_2\left(\frac{|Z|}{\sqrt{3}\tau };\sqrt{3}L\right)={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(|Z|\ge \frac{\tau }{L}{h}_2^{-1}\left(\frac{L^2}{2}3t\right)\right)}{e}^{t}dt\le {\displaystyle {\int }_0^{\mathrm{\infty }}2\exp (-3t)\exp (t)dt=1.}$$

Hence we conclude that $\Vert Z{\Vert }_{{\mathrm{\Psi }}_2(·;\sqrt{3}L)}\le \sqrt{3}\tau$.

Corollary 3.1

1. If X ~ Poisson(ν), then $\Vert X-\nu {\Vert }_{{\mathrm{\Psi }}_2(·;\sqrt{6/\nu })}\le \sqrt{6/\nu }$.

2. If X₁, …, X_n are i.i.d. Bernoulli(p), then

   $$\Vert \sqrt{n}({\overline{X}}_n-p){\Vert }_{{\mathrm{\Psi }}_2(·;\sqrt{2}/(\sqrt{np(1-p)}))}\le \sqrt{6p(1-p)}.$$

3. If X ~ N(0, 1), then $\Vert X{\Vert }_{{\mathrm{\Psi }}_h(·;L)}\le \sqrt{6}$ for every L > 0. By taking the limit on L ↘ 0 and noting that Ψ₂·(z; L) → Ψ₂(z) ≡ exp(z²) − 1 as L ↘ 0 this yields $\Vert X{\Vert }_{{\mathrm{\Psi }}_2}\le \sqrt{6}$. In this case it is known that $\Vert X{\Vert }_{{\mathrm{\Psi }}_2}=\sqrt{8/3}$. (See van der Vaart and Wellner (1996), Exercise 2.2.1, page 105.)

Now for an inequality paralleling Lemma 2.3 for the Bernstein-Orlicz norm:

Lemma 3.3

Let τ and L be constants, and let Z₁, …, Z_m be random variables satisfying ${\max }_{1\le j\le m}\Vert Z_j{\Vert }_{{\mathrm{\Psi }}_2(·;L)}\le \tau$. Then

$$E\{\underset{1\le j\le m}{\max }|Z_j|\}\le \tau {\mathrm{\Psi }}_2^{-1}(m;L)\le 2\tau L\frac{\log (1+m)}{\log \log (1+m)}if{L}^2\ge 2\mathit{and}\log (1+m)\ge 5.$$

Furthermore,

$$E\{\underset{1\le j\le m}{\max }|Z_j|\}\le \tau {\mathrm{\Psi }}_2^{-1}(m;L)\le 2\tau \left\{L\frac{\log (1+m)}{\log \log (1+m)}+\sqrt{\log (1+m)}\right\}$$

for all m such that log(1 + m) ≥ 5 (or m ≥ e⁵ − 1).

Remark 3.1

The point of this last bound is that it gives an explicit trade-off between the Gaussian component (the term $\sqrt{\log (1+m)}$) and the Poisson component (the term log(1 + m)/log log(1 + m)) governed by a Bennett type inequality. In contrast, the bounds obtained by van de Geer and Lederer (2013) yield a trade-off between the Gaussian world and the sub-exponential world governed by a Bernstein type inequality.

Proof

We write Ψ_2,L ≡ Ψ₂(·; L). Let c>τ. Then by Jensen’s inequality

$$E\left\{\underset{1\le j\le m}{\max }|Z_j|\right\}\le c{\mathrm{\Psi }}_{2,L}^{-1}\left(E\{{\mathrm{\Psi }}_{2,L}(\underset{1\le j\le m}{\max }|Z_j|/c)\}\right)=c{\mathrm{\Psi }}_{2,L}^{-1}\left(E\{\underset{1\le j\le m}{\max }{\mathrm{\Psi }}_{2,L}(|Z_j|/c)\}\right)\le c{\mathrm{\Psi }}_{2,L}^{-1}\left({\displaystyle \sum _{j=1}^{m}E{\mathrm{\Psi }}_{2,L}(|Z_j|/c)}\right)\le c{\mathrm{\Psi }}_{2,L}^{-1}\left(m\underset{1\le j\le m}{\max }E{\mathrm{\Psi }}_{2,L}(|Z_j|/c)\right).$$

Therefore,

$$E\left\{\underset{1\le j\le m}{\max }|Z_j|\right\}\le \underset{c\searrow \tau }{\lim }\left\{c{\mathrm{\Psi }}_{2,L}^{-1}\left(m\underset{1\le j\le m}{\max }E{\mathrm{\Psi }}_{2,L}(|Z_j|/c)\right)\right\}$$ (3.3)

$$\tau {\mathrm{\Psi }}_{2,L}^{-1}(m)=\frac{\tau }{L}{h}_2^{-1}\left(\frac{L^2}{2}\log (1+m)\right).$$ (3.4)

The remaining claims follow from Proposition 3.3.

Here are analogues of Lemmas 4 and 5 of van de Geer and Lederer (2013).

Lemma 3.4

Let Z₁, …, Z_m be random variables satisfying

$$\underset{1\le j\le m}{\max }\Vert Z_j{\Vert }_{{\mathrm{\Psi }}_2(·,L)}\le \tau$$ (3.5)

for some L and τ. Then, for all t > 0

$$P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \frac{\tau }{L}(h_2^{-1}(L^{2}t/2)+h_2^{-1}(L^2\log (1+m)/2))\right)\le 2e^{-t}.$$

Proof

For any a > 0 and t > 0 concavity of $h_2^{-1}$ together with $h_2^{-1}(0)=0$ imply that

$$h_2^{-1}(a)+h_2^{-1}(t)\ge h_2^{-1}(a+t).$$

Therefore, by using a union bound and Lemma 3.1

$$P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \frac{\tau }{L}(h_2^{-1}(L^{2}t/2)+h_2^{-1}(L^2\log (1+m)/2))\right)\le P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \frac{\tau }{L}(h_2^{-1}(2^{-1}{L}^2(t+\log (1+m))))\right)\le {\displaystyle \sum _{j=1}^{m}P\left(|Z_j|\ge \frac{\tau }{L}(h_2^{-1}(2^{-1}{L}^2(t+\log (1+m))))\right)}\le 2m\exp (-(t+\log (1+m)))=2\frac{m}{m+1}{e}^{-t}\le 2e^{-t}.$$

Lemma 3.5

Let Z₁, …, Z_m be random variables satisfying (3.5). Then

$${\Vert {\left(\underset{1\le j\le m}{\max }|Z_j|-\frac{\tau }{L}{h}_2^{-1}\left(\frac{L^2}{2}\log (1+m)\right)\right)}_{+}\Vert }_{{\mathrm{\Psi }}_2(·;L)}\le \sqrt{3}\tau .$$

Proof

Let

$$Z\equiv {\left(\underset{1\le j\le m}{\max }|Z_j|-\frac{\tau }{L}{h}_2^{-1}\left(\frac{L^2}{2}\log (1+m)\right)\right)}_{+}.$$

Then Lemma 3.4 implies that

$$P\left(Z\ge \frac{\tau }{L}{h}_2^{-1}\left(\frac{L^{2}t}{2}\right)\right)\le P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \frac{\tau }{L}\left(h_2^{-1}\left(\frac{L^2}{2}\log (1+m)\right)+h_2^{-1}\left(\frac{L^{2}t}{2}\right)\right)\right)\le 2e^{-t}.$$

Then the conclusion follows from Lemma 3.2.

4 Prokhorov’s “Arcsinh” Exponential Bound and Orlicz Norms

Another important exponential bound for sums of independent bounded random variables is due to Prokhorov (1959). As will be seen below, Prokhorov’s bound involves another function h₄ (rather than h₂ of Bennett’s inequality) given by

$$h_4(x)=(x/2)\text{arcsinh}(x/2)=(x/2)\log \left(x/2+\sqrt{1+{(x/2)}^2}\right).$$ (4.1)

Suppose that X₁, …, X_n are independent random variables with E(X_j) = μ_j and |X_j − μ_j| ≤ b for some b > 0. Let S_n = X₁ + ⋯ + X_n, and set $\mu \equiv n^{-1}{\displaystyle {\sum }_{j=1}^n{\mu }_j}$, ${\sigma }_n^2\equiv n^{-1}\mathit{\text{Var}}(S_n)$. Prokhorov’s “arcsinh” exponential bound is as follows:

Proposition 4.1

(Prokhorov) If the X_j’s satisfy the above assumptions, then

$$P(S_n-n\mu \ge z)\le \exp \left(-\frac{z}{2b}\text{arcsinh}\left(\frac{zb}{2{\sigma }_n^2}\right)\right).$$

Equivalently, with ${\sigma }_n^2\equiv n^{-1}\mathit{\text{Var}}(S_n)$ and h₄(x) ≡ (x/2)arcsinh(x/2),

$$P(|\sqrt{n}({\overline{X}}_n-\mu )|\ge z)=2\exp \left(-\frac{z\sqrt{n}}{2b}\text{arcsinh}\left(\frac{zb}{2\sqrt{n}{\sigma }_n^2}\right)\right)=2\exp \left(-\frac{n{\sigma }_n^2}{b^2}\left(\frac{zb}{2\sqrt{n}{\sigma }_n^2}\right)\text{arcsinh}\left(\frac{zb}{2\sqrt{n}{\sigma }_n^2}\right)\right)\equiv 2\exp \left(-\frac{2{\sigma }_n^2}{b^2}{h}_4\left(\frac{zb}{\sqrt{n}{\sigma }_n^2}\right)\right).$$ (4.2)

See e.g. Prokhorov (1959), Stout (1974), de la Peña and Giné (1999), Johnson et al. (1985), & Kruglov (2006). Johnson et al. (1985) use Prokhorov’s inequality to control Orlicz norms for functions Ψ of the form Ψ(x) = exp(ψ(x)) with ψ(x) ≡ x log(1+x) and use the resulting inequalities to show that the optimal constants D_p in Rosenthal’s inequalities grow as p/log(p).

Kruglov (2006) gives an improvement of Prokhorov’s inequality which involves replacing h₄ by

$$h_5(x)\equiv x\text{arcsinh}(x/2)-2\left(\cosh (\text{arcsinh}(x/2))-1\right).$$

Note that Prokhorov’s inequality is of the same form as Bennett’s inequality (3.1) in Proposition 3.1, but with Bennett’s h₂ replaced by Prokhorov’s h₄.

Thus we want to compare Prokhorov’s inequality (and Kruglov’s improvement thereof) to Bennett’s inequality. As can be seen from the above development, this boils down to comparison of the functions h₂, h₄, and h₅. The following lemma makes a number of comparisons and contrasts between the functions h₂, h₄, and h₅.

Lemma 4.1

(Comparison of h₂, h₄, and h₅)

• (i)(a)h₂(x) ≥ h₅(x) ≥ h₄(x) for all x ≥ 0.
• (i)(b) $h_2^{-1}(y)\le h_5^{-1}(y)\le h_4^{-1}(y)$ for all y ≥ 0.
• (ii)(a)

  h₂(x) ≥ (x/2) log(1 + x) ≥ (x/2) log(1 + x/2) for all x ≥ 0.

• (ii)(b)

  h₄(x) ≥ (x/2) log(1 + x/2) for all x ≥ 0.

• (ii)(c)

  h₅(x) ≥ (x/2) log(1 + x/2) for all x ≥ 0.

• (iii)(a)

  h₂(x) ~ 2⁻¹x² as x ↘ 0; h₂(x) ~ x log(x) as x → ∞.

• (iii)(b)

  h₄(x) ~ 4⁻¹x² as x ↘ 0; h₄(x) ~ (1/2)x log(x) as x → ∞.

• (iii)(c)

  h₅(x) ~ 4⁻¹x² as x ↘ 0; h₅(x) ~ x log(x) as x → ∞.

• (iii)(d)

  h₂(x) − h₄(x) ~ x²/4 as x ↘ 0; h₂(x) − h₄(x) ~ (1/2)x log x as x→ ∞.

• (iii)(e)

  h₂(x) − h₅(x) ~ x²/4 as x ↘ 0; h₂(x) − h₅(x) ~ log x as x→ ∞.

• (iv)(a)
  h₂(x) = 2⁻¹x²ψ₂(x) where

  $${\psi }_2(x)\ge \{\frac{x^{-1}\log (1+x),}{{(1+x/3)}^{-1}.}$$
• (iv)(b)
  h₄(x) = 4⁻¹x²ψ₄(x) where

  $${\psi }_4(x)\ge \{\begin{array}{ll}2x^{-1}\log (1+x/2),\hfill & \mathit{\text{for}}x\ge 0\hfill \\ (1/2)/{(1+x/2)}^{-1},\hfill & \mathit{\text{for}}x\ge 0\hfill \\ (1-\delta )/(1+x/2),\hfill & \mathit{\text{for}}x\le 2{\delta }^{1/2}/{(1/2-\delta )}^{1/2}.\hfill \end{array}$$
• (iv)(c)
  h₅(x) = 4⁻¹x²ψ₅(x) where

  $${\psi }_5(x)\ge \{\begin{array}{ll}2x^{-1}\log (1+x/2),\hfill & \mathit{\text{for}}x\ge 0\hfill \\ 1/{(1+x/2)}^{-1},\hfill & \mathit{\text{for}}x\ge 0.\hfill \end{array}$$

Proof

(i) We first prove that h₂(x) ≥ h₄(x). Let g(x) = h₂(x) − h₄(x); thus

$$g(x)=(1+x)\log (1+x)-x-(x/2)\log (x/2+\sqrt{1+{(x/2)}^2}).$$

Then g(0) = 0 and

$$g^{\prime }(x)=\log (1+x)-\frac{1}{2}\log \left(x/2+\sqrt{1+{(x/2)}^2}\right)-\frac{x/4}{\sqrt{1+{(x/2)}^2}}$$

also has g′(0) = 0. Note that $\sqrt{1+{(x/2)}^2}\le 1+x/2$ and hence $x/2+\sqrt{1+{(x/2)}^2}\le 1+x$. Thus

$$\log ((x/2)+\sqrt{1+{(x/2)}^2})\le \log (1+x),$$ (4.3)

and hence

$$g^{\prime }(x)\ge \log (1+x)-(1/2)\log (1+x)-\frac{x/4}{\sqrt{1+{(x/2)}^2}}=(1/2)\log (1+x)-\frac{x/4}{\sqrt{1+{(x/2)}^2}},$$

and it suffices to show that the right side is ≥ 0 for all x. Thus we let

$$m(x)\equiv g^{\prime }(x)=\frac{1}{2}\log (1+x)-\frac{1}{2}\frac{x/2}{\sqrt{1+{(x/2)}^2}}=\frac{1}{2}\left\{\log (1+x)-\frac{x/2}{\sqrt{1+{(x/2)}^2}}\right\}.$$

Let $\overline{m}(x)\equiv 2m(2x)=\log (1+2x)-\frac{x}{\sqrt{1+x^2}}$. Then $\overline{m}(0)=0$ and we compute

$$\begin{gathered} {\overline{m}}^{\prime }(x)=\frac{2}{1+2x}-\frac{1}{\sqrt{1+x^2}}+\frac{x^2}{{(1+x^2)}^{3/2}} \\ =\frac{2}{1+2x}-\frac{1}{\sqrt{1+x^2}}\left(1-\frac{x^2}{(1+x^2)}\right) \\ =\frac{2{(1+x^2)}^{3/2}-(1+2x)}{(1+2x){(1+x^2)}^{3/2}}\equiv \frac{j(x)}{(1+2x){(1+x^2)}^{3/2}} \end{gathered}$$

so that ${\overline{m}}^{\prime }(0)=1$ and the numerator, j, is easily seen to be non-negative since (1 + x²)^3/2 ≥ 1 + x² implies 2(1 + x²)^3/2 ≥ 2(1 + x²) ≥ 1 + 2x for all x ≥ 0. Thus h₂(x) ≥ h₄(x).

Kruglov (2006) shows that h₅(x) ≥ h₄(x). Now we show that h₂(x) ≥ h₅(x). Note that with g(x) ≡ h₂(x) − h₅(x),

$$g^{\prime }(x)=\log (1+x)-\text{arcsinh}(x/2)$$

has g′(x) = 0 and g′(x) ≥ 0 (as was shown above in (4.3)). Thus $g(x)={\displaystyle {\int }_0^x{g}^{\prime }(v)dv\ge 0}$.

• (i)(b)The inequalities for the inverse functions follow immediately from the inequalities for the functions themselves in (i)(a).
• (ii)(a)
  To show that the first inequality holds, consider

  $$g(x)\equiv h_2(x)-(x/2)\log (1+x)=(1+x)\log (1+x)-x-(x/2)\log (1+x)=(1+x/2)\log (1+x)-x.$$

  Then g(0) = 0 and

  $$g^{\prime }(x)=\frac{1}{2}\log (1+x)+\frac{1+x/2}{1+x}-1=\frac{1/2}{1+x}\{(1+x)\log (1+x)-x\}=\frac{1/2}{1+x}{h}_2(x)\ge 0.$$

  Thus g′ (0) = 0 and $g(x)={\displaystyle {\int }_0^x{g}^{\prime }(y)dy\ge 0}$. The second inequality in (ii)(a) is trivial.
• (ii)(b)

  This follows easily from $\text{arcsinh}(v)=\log (v+\sqrt{1+v^2})\ge \log (v+1)$ for all v ≥ 0.

• (ii)(c)

  This follows from (i)(a) and (ii)(b).

• (iii)(a)

  This follows from ψ₂(x) ≡ ψ(x) → 1 as x ↘ 0; see Proposition 11.1.1, page 441, Shorack and Wellner (1986).

• (iii)(b)
  Now

  $$h_4^{\prime }(x)=\frac{x}{4\sqrt{1+{(x/2)}^2}}+\frac{1}{2}\text{arcsinh}(x/2),$$

  with $h_4^{\prime }(0)=0$, and

  $$h_4^{″}(x)=\frac{1}{2\sqrt{1+{(x/2)}^2}}-\frac{x^2}{16{(1+{(x/2)}^2)}^{3/2}}=\frac{8+x^2}{2{(4+x^2)}^{3/2}}$$

  with $h_4^{″}(0)=1/2$. Therefore

  $$h_4(x)=h_4^{″}(x^{\ast })\frac{x^2}{2}\text{for some}0\le x^{\ast }\le x$$

  and

  $$\frac{4}{x^2}{h}_4(x)=2h_4^{″}(x^{\ast })\to 1\text{as }x\ge x^{\ast }\searrow 0.$$
• (iii)(c)
  Now

  $$h_5^{\prime }(x)=\text{arcsinh}(x/2),$$

  $$h_5^{″}(x)=\frac{1}{2\sqrt{1+{(x/2)}^2}}\to \frac{1}{2}\text{as }x\searrow 0,$$

  where $h_5^{\prime }(x)=0$ and $h_5^{″}$ is decreasing. Thus $h_5(x)=(x^2/2)h_5^{″}(x^{\ast })$ for some 0 ≤ x ≤ x* and we conclude that 4x⁻²h₅(x) → 1 as x ≥ x* ↘ 0.
• (iv)(a)

  The first part is a restatement of (ii)(a). The second part follows from Eq. 2.6: h₂(x) = h(1 + x) ≥ 9h₀(x) = x²/(2(1 + x/3)), and the claim follows by definition of ψ₂.

• (iv)(b)
  The first inequality is a restatement of (ii)(b). The second inequality follows since $h_4(x)=h_4^{″}(x^{\ast })$ where $x\mapsto h_4^{″}(x)$ is decreasing, so

  $$\frac{4}{y^2}{h}_4(x)=2h_4^{″}(x^{\ast })\ge 2h_4^{″}(x)=\frac{1}{\sqrt{1+{(x/2)}^2}}-\frac{x^2}{8{(1+{(x/2)}^2)}^{3/2}}=\frac{1}{\sqrt{1+{(x/2)}^2}}·\frac{1+x^2/8}{1+x^2/4}\ge \frac{1/2}{\sqrt{1+{(x/2)}^2}}\ge \frac{1/2}{1+x/2}.$$

  To prove the third inequality, note that

  $$\frac{1+x^2/8}{1+x^2/4}\ge c$$

  holds if 1 + x²/8 ≥ c(1 + x²/4), or if 1 − c ≥ (x²/4)(c − 1/2). Then rearrange and take c = (1 − δ) for δ ∈ (0, 1/2).
• (iv)(c)
  The first inequality follows from (ii)(c). The second inequality follows by arguing as in (iv)(b), but now without the complicating second factor: note that

  $$\frac{4}{x^2}{h}_5(x)=2h_5^{″}(x^{\ast })\ge 2h_5^{″}(x)=\frac{1}{\sqrt{1+{(x/2)}^2}}\ge \frac{1}{1+x/2}$$

  since $h_5^{″}$ is decreasing.

Discussion

1. Even though Kruglov’s inequality improves on Prokhorov’s inequality, (ia) of Lemma 4.1 shows that Bennett’s inequality dominates both Kruglov’s improvement of Prokhorov’s inequality and Prokhorov’s inequality itself: h₂ ≥ h₅ ≥ h₄.

2. (ii) of Lemma 4.1 shows that all three of the inequalities, Bennett, Kruglov, and Prokhorov, are based on functions h₂, h₅, and h₄ which are bounded below by (x/2) log(1+x/2) for all x ≥ 0. On the other hand, (ii)(d) shows that both h₂ and h₅ are very nearly equivalent for large x, but that although h₄ grows at the same x log x rate as h₂ and h₅, h₄ is smaller by a multiplicative factor of 1/2 as x → ∞.

3. (iii)(a-c) of Lemma 4.1 shows that h₂(x) ~ x²/2 as x ↘ 0 while h_k(x) ~ x²/4 for both h₅ and h₄; thus h₂(x) is larger at x = 0 by a factor of 2. Furthermore, the difference h₂ − h₄ is of order (1/2)x log x as x→∞, while the difference h₂ − h₅ is only of order log x as x → ∞.

4. (iv) of Lemma 4.1 re-expresses the behavior of the Kruglov and Prokhorov inequalities for small values of x in terms of the corresponding ψ_k functions. The upshot of all of these comparisons is that Bennett’s inequality dominates both the Kruglov and Prokhorov inequalities. Figures 1–2 give graphical versions of these comparisons as well as comparisons to the Bernstein type h–functions h₀ and h₁.

Figure 1.

Comparison of the h_k functions h₀, h₁, h₂, h₄, and h₅. The plot shows the functions h_k. The function h₀ is plotted in magenta (tiny dashing), h₁ in blue (medium dashing), h₂ in red (no dashing), h₄ in purple (large dashing), and h₅ is plotted in black (medium dashing). For values of the argument larger than ≈ 1.4 h₂ > h₅ > h₄ ≫ h₁ > h₀ (and all are below h₂), while for values of the argument smaller than ≈ 1, h₂ > h₁ > h₀ ≫ h₅ > h₄

Figure 2.

The plot depicts (with the same colors and dashing as in Fig. 1) the ratios x ↦ h_k(x)/(x²/2) ≡ ψ_k(x) for k ∈ {0, 1, 2, 4, 5}. This figure illustrates our finding that the Prokhorov type h–functions are smaller by a factor of 1/2 at x = 0, while they again dominate the Bernstein type h–functions for larger values of x, with the cross-overs occurring again between 1 and 1.4

5 Comparisons with Some Results of Talagrand

Our goal in this section is to give comparisons with some results of Talagrand (1989^{, 1994}), especially his Theorem 3.5, page 45, and Proposition 6.5, page 58.

Talagrand (1994) defines a function φ_L,S as follows:

$${\phi }_{L,S}(x)\equiv \{\begin{array}{ll}\frac{x^2}{L^{2}S},\hfill & \text{if }x\le LS,\hfill \\ \frac{x}{L}{\left(\log \left(\frac{ex}{LS}\right)\right)}^{1/2}\hfill & \text{if }x\ge LS.\hfill \end{array}$$

Because of the square-root on the log term, this can be regarded as corresponding to a “sub - Bennett” type exponential bound. One of the interesting properties of φ_L,S established by Talagrand (1994) is given in the following lemma:

Lemma 5.1

There is a number K(L) depending on L only such that

$${\phi }_{L,S}(K(L)x\sqrt{S})\ge 11x^2\mathit{\text{for all}}x\le K(L)\sqrt{S}.$$

This is Lemma 3.6 of Talagrand (1994) page 47. Talagrand uses this Lemma to develop a Kiefer-type inequality: see also van der Vaart and Wellner (1996), Corollary A.6.3. In the basic Kiefer type inequality for Binomial random variables, van der Vaart and Wellner (1996), Corollary A.6.3, it follows that

$$P(\sqrt{n}|{\overline{Y}}_n-p|\ge z)\le 2\exp (-11z^2)$$

for log(1/p) − 1 ≥ 11; i.e. for p ≤ e⁻¹².

A similar fact holds for any exponential bound of the Bennett type under a certain boundedness hypothesis. Suppose that

$$P(|Z|>z)\le 2\exp \left(-\frac{z^2}{2\tau }\psi \left(\frac{Lz}{\tau }\right)\right)$$

and that P (|Z| ≥ v) = 0 for all v ≥ C. Then, since ψ is decreasing, for z ≤ C

$$P(|Z|>z)\le 2\exp \left(-\frac{z^2}{2\tau }\psi \left(\frac{LC}{\tau }\right)\right)=2\exp \left(-\frac{z^2}{2LC}\frac{LC}{\tau }\psi \left(\frac{LC}{\tau }\right)\right)\le 2\exp \left(-\frac{z^2}{LC}\log \left(\frac{LC}{e\tau }\right)\right)$$

where the log term can be made arbitrarily large by choosing τ sufficiently small. Here the second inequality follows from the fact that

$$x\psi (x)\ge 2\log (x/e)=2(\log x-1).$$ (5.1)

Proof. of Eq. 5.1

Proof. of Eq. 5.1: Since ψ(x) = 2x⁻²h(1 + x) where h(x) = x(log x − 1) + 1, we can write, with $\underset{\_}{\psi }(x)\equiv 2x^{-2}\log (x/e)$,

$$\frac{1}{2}(x\psi (x)-\underset{\_}{\psi }(x))=\frac{1}{x}\{(1+x)\log (1+x)-x\}-(\log x-1)=\frac{1+x}{x}\log (1+x)-\log x=\log \left(\frac{1+x}{x}\right)+\frac{1}{x}\log (1+x)$$

where both terms are clearly non-negative.

Now we consider another basic inequality due to Talagrand (1994). Suppose that

$$\theta :2^{\mathcal{X}}\cap \{C\in 2^{\mathcal{X}}:|C|<\mathrm{\infty }\}\mapsto \mathrm{ℝ}$$

satisfies the following three properties:

1. C ⊂ D implies that θ(C) ≤ θ(D) for C, $D\in 2^{\mathcal{X}}$.

2. θ(C ∪ D) ≤ θ(C) + θ(D).

3. θ(C) ≤ |C| = #(C).

Then if X₁, …, X_n are i.i.d. P non-atomic on $(\mathcal{X},A)$ and Z ≡ θ({X₁, …, X_n}), for some universal constant K₂ we have, for z ≥ K₂E(Z),

$$P(Z\ge z)\le \exp \left(-\frac{z}{K_2}\log \left(\frac{ez}{K_{2}E(Z)}\right)\right).$$

As noted by Talagrand (1994), this follows from an isoperimetric inequality established in Talagrand (1989), but it is also a consequence of results of Talagrand (1991^{, 1995}). Here we simply note that it can be rephrased as a Bennett type inequality: for all z ≥ K₂E(Z)

$$P(Z\ge z)\le \exp \left(-E(Z)h\left(1+\frac{z}{K_{2}E(Z)}\right)\right).$$

This follows by simply checking that

$$E(Z)h\left(1+\frac{z}{K_{2}E(Z)}\right)\le \frac{z}{K_{2}E(Z)}\log \left(\frac{ez}{K_{2}E(Z)}\right)$$

for z ≥ K₂E(Z).

Also see Ledoux (2001), Theorem 7.5, page 142 and Corollary 7.8, page 148; Massart (2000), and Boucheron et al. (2013), Theorem 6.12, page 182.

One further remark seems to be in order: Talagrand (1989) Theorem 2 and Proposition 12, shows that Orlicz norms of the Bennett type are “too large” to yield nice generalizations of the classical Hoffmann-Jørgensen inequality in the setting of sums of independent bounded sequences in a general Banach space. This follows by noting that Talagrand’s condition (2.11) fails for the Bennett-Orlicz norm Ψ₂(·, L) as defined in Eq. 3.2.

Appendix 1: Lambert’s Function W; Inverses of h and h₂

Let h(x) ≡ x(log x − 1) + 1 and h₂(x) ≡ h(1 + x) for x ≥ 0. The function h is convex, decreasing on [0, 1], increasing on [1, ∞), with h(1) = 0; see Shorack and Wellner (1986), page 439. The Lambert, or product log function, W (see e.g. Corless et al. (1996) and satisfies W (x)e^W(x) = x for x ≥ −1/e. As noted by Boucheron et al. (2013), problem 2.18, the inverse functions h⁻¹ (for the function h: [1, ∞) → [0, ∞)) and $h_2^{-1}$ (for the function h₂: [0, ∞) → [0, ∞)) can be expressed in terms of the function W. Here are some facts about W:

Fact 1

W: [−1/e, ∞) ↦ ℝ is multi-valued on [−1/e, 0) with two branches W₀ and W₋₁ where W₀(x) > 0, W₋₁(x) < 0, and W₀(−1/e) = −1 = W₋₁(−1/e).

Fact 2

W₀ is monotone increasing on [−1/e, ∞) with W (0) = 0 and W′ (0) = 1.

See Roy and Olver (2010), section 4.13, page 111; and Corless et al. (1996).

In the following we simply write W for W₀. The following lemma shows that the inverses of the functions h and h₂ can be expressed in terms of W.

Lemma 6.1

(h and h₂ inverses in terms of W)

1. For y ≥ 0

   $$h^{-1}(y)=e\frac{(y-1)/e}{W((y-1)/e)}\ge 1.$$ (6.1)
2. For y ≥ 0

   $$h_2^{-1}(y)=h^{-1}(y)-1=e\frac{(y-1)/e}{W((y-1)/e)}-1\ge 0.$$ (6.2)

Proof

If h⁻¹ is as in the display we have, since h(x) = x(log x − 1) + 1,

$$h(h^{-1}(y))=\frac{e(y-1)/e}{W((y-1)/e)}\left(\log \left(\frac{(y-1)/e}{W((y-1)/e)}\right)+1-1\right)+1=\frac{e(y-1)/e}{W((y-1)/e)}W((y-1)/e)+1\text{since }{e}^{W(x)}=\frac{x}{W(x)}=y-1+1=y.$$

Thus Eq. 6.1 holds. Then Eq. 6.2 follows immediately.

In view of Lemma 6.1, the following lower bounds on the function W will be useful in deriving upper bounds on h⁻¹ and $h_2^{-1}$.

Lemma 6.2

(A lower bound for W) For z > 0

$$W(z)\ge \frac{1}{2}\log (ez)=2^{-1}(1+\log z).$$ (6.3)

Proof

We first prove (6.3) for z ≥ 1/e. Since W(z) is increasing for z ≥ 0, the claimed inequality is equivalent to

$$z=W(z)e^{W(z)}\ge \frac{1}{2}\log (ez)\exp ((1/2)\log (ez))={(ez)}^{1/2}\log ({(ez)}^{1/2})\equiv y\log y,$$

for ez ≥ 1 where y ≡ (ez)^1/2. But then the last display is equivalent to

$$e^{-1}{y}^2\ge y\log y\text{for}y\ge 1$$

or

$$g(y)\equiv y^2-ey\log y\ge 0\text{for all }y\ge 1.$$

Now g(1) = 0, g(e) = 0, and g′(y) = 2y − e − e log y has g′(1) = 2 − e < 0, g′(e) = 0, and g′(y) > 0 for y > e with g″(y) = 2 − e/y, we find that g″(e) = 2 − e/e = 1 > 0. Thus the claimed bound holds for z ≥ 1/e. For 0 ≤ z < 1/e the bound holds trivially since W (z) ≥ 0 while 2⁻¹ log(ez) < 0.

Combining Lemma 6.1 with the lower bounds for W given in Lemma 6.2 yields the following upper bounds for h⁻¹ and $h_2^{-1}$. The second and third parts of the following lemma are motivated by the fact that h₂(x) = h(1 + x) ≡ (x²/2)ψ(x) where ψ(x) ↗ 1 as x ↘ 0; see Shorack and Wellner (1986), Proposition 4.4.1, page 441.

Lemma 6.3

(Upper bounds for h⁻¹ and $h_2^{-1}$)

1. For y > 1 + e

   $$h^{-1}(y)\le 2\frac{y-1}{\log (y-1)}.$$ (6.4)
2. For y > 1 + e,

   $$h_2^{-1}(y)\le 2\frac{y-1}{\log (y-1)}-1.$$ (6.5)
3. For 0 ≤ y ≤ 9c⁻²(c²/2 − 1)² with $c>\sqrt{2}$,

   $$h_2^{-1}(y)\le c\sqrt{y}.$$ (6.6)

   In particular, with c = 2, the bound holds for 0 ≤ y ≤ 9/4, and with c = 2.2, the bound holds for 0 ≤ y ≤ 1 + e.
4. For 0 < y < ∞,

   $$h_2^{-1}(y)\le \{\begin{array}{ll}2.2\sqrt{v}\hfill & y\le 1+e,\hfill \\ 2\frac{y-1}{\log (y-1)}-1\hfill & y>1+e.\hfill \end{array}$$ (6.7)

Proof

1. Follows from (i) of Lemma 6.1 together with Lemma 6.2. Note that g(x) ≡ x/log(x) ≥ e and g is increasing for x ≥ e.

2. follows from (ii) of Lemma 6.1 and Lemma 6.2.

3. To show that Eq. 6.6 holds, note that the inequality is equivalent to $y\le h_2(c\sqrt{y})$, and hence, by taking $x\equiv c\sqrt{y}$, to the inequality

   $$x^2/c^2\le h_2(x)=h(1+x)=\frac{x^2}{2}\psi (x)$$

   where ψ(x) ≡ (2/x²)h(1 + x) ≥ 1/(1 + x/3) by Lemma 4.1 (iva) (or by (10) of Proposition 11.4.1, Shorack and Wellner (1986) page 441). But then we have

   $$h_2(x)=h(1+x)\ge \frac{x^2}{2}\frac{1}{1+x/3}\ge \frac{x^2}{c^2}$$

   where the last inequality holds if 0 ≤ x ≤ 3(c²/2 − 1). Hence the inequality in (iii) holds for 0 ≤ y ≡ x²/c² ≤ 9(c²/2 − 1)²/c−2. Finally, (iv) holds by combining the bounds in (ii) and (iii).

Appendix 2: General versions of Lemmas 1-5

Now consider Young functions of the form Ψ = e^ψ − 1 where ψ is assumed to be convex and nondecreasing with ψ(0) = 0. (Note that we have changed notation in this section: the functions h and h_j for j ∈ {0, 1, 2, 4, 6} in Sections 1–6 are denoted here by ψ.) Our goal in this section is to give general versions of Lemmas 1 - 5 of van de Geer and Lederer (2013) and Section 3 above. The advantage of this formulation is that the resulting lemmas apply to all the special cases treated in Sections 2 and 3 and more.

Lemma 7.1

Suppose that τ ≡ ‖Z‖_Ψ < ∞. Then for all t > 0

$$P(|Z|>\tau {\psi }^{-1}(t))\le 2e^{-t}.$$

For the general version of Lemma 2 we consider a scaled version of Ψ as follows:

$$\mathrm{\Psi }(z;L)\equiv {\mathrm{\Psi }}_L(z)\equiv \exp \left(\frac{2}{L^2}\psi (Lz)\right)-1.$$ (7.1)

Lemma 7.2

Suppose that for some τ > 0 and L > 0

$$P\left(|Z|\ge \frac{\tau }{L}{\psi }^{-1}(L^{2}t/2)\right)\le 2e^{-t}\mathit{\text{for all}}t>0.$$

Then $\Vert Z{\Vert }_{\mathrm{\Psi }(·;\sqrt{3}L)}\le \sqrt{3}\tau$.

Lemma 7.3

Suppose that Ψ is non-decreasing, convex, with Ψ(0)=0. Suppose that Z₁, …, Z_m are random variables with max_1≤j≤m ‖Z_j‖_Ψ≡τ <∞. Then

$$E\left(\underset{1\le j\le m}{\max }|Z_j|\right)\le \tau {\mathrm{\Psi }}^{-1}(m).$$

Lemma 7.4

Suppose that Ψ is non-decreasing, convex, with Ψ(0)=0. Suppose that Z₁, …, Z_m are random variables with max_1≤j≤m ‖Z_j‖_Ψ≡τ <∞. Then

$$P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \tau ({\psi }^{-1}(\log (1+m))+{\psi }^{-1}(t))\right)\le 2e^{-t}.$$

Lemma 7.5

Suppose that Ψ is non-decreasing, convex, with Ψ(0)=0. Suppose that Z₁, …, Z_m are random variables with max_1≤j≤m‖Z_j‖_Ψ ≡τ <∞. Then

$${\Vert {\left(\underset{1\le j\le m}{\max }|Z_j|-\tau {\mathrm{\Psi }}^{-1}(m)\right)}_{+}\Vert }_{\mathrm{\Psi }(·;\sqrt{6})}\le \sqrt{6}\tau .$$

Proof of Lemma 7.1

For all c > ‖Z‖_Ψ

$$P(|Z|/c>{\psi }^{-1}(t))=P(\psi (|Z|/c)>t)=P(e^{\psi (|Z|/c)}-1>e^t-1)=P(\mathrm{\Psi }(|Z|/c)>e^t-1)\le (E\mathrm{\Psi }(|Z|/c)+1)e^{-t}.$$

Thus letting c ↘ τ yields

$$P(|Z|/\tau >{\psi }^{-1}(t))=\underset{c\searrow \tau }{\lim }P(|Z|/c>{\psi }^{-1}(t))\le \underset{c\searrow \tau }{\lim }(E\mathrm{\Psi }(|Z|/c)+1)e^{-t}\le 2e^{-t}.$$

Proof of Lemma 7.2

Let α, β > 0. We compute

$$E\mathrm{\Psi }\left(\frac{|Z|}{\alpha \tau };\beta L\right)={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(\mathrm{\Psi }\left(\frac{|Z|}{\alpha \tau };\beta L\right)\ge v\right)dv}={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(\frac{2}{{\beta }^2{L}^2}\psi \left(\frac{\beta L|Z|}{\alpha \tau }\right)\ge \log (1+v)\right)dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(\frac{\beta L|Z|}{\alpha \tau }\ge {\psi }^{-1}\left(\frac{{\beta }^2{L}^2}{2}\log (1+v)\right)\right)}dv={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(|Z|\ge \frac{\alpha \tau }{\beta L}{\psi }^{-1}\left(\frac{{\beta }^2{L}^2}{2}\log (1+v)\right)\right)dv}={\displaystyle {\int }_0^{\mathrm{\infty }}P\left(|Z|\ge \frac{\alpha \tau }{\beta L}{\psi }^{-1}\left(\frac{{\beta }^2{L}^2}{2}t\right)\right)}{e}^{t}dt}\le {\displaystyle {\int }_0^{\mathrm{\infty }}2e^{-3t}{e}^{t}dt=1}$$

by choosing $\alpha =\beta =\sqrt{3}$.

Proof of Lemma 7.3

Let c > τ. Then by Jensen’s inequality and convexity of Ψ

$$\mathrm{\Psi }\left(\frac{E\{{\max }_{1\le j\le m}|Z_j|\}}{c}\right)\le E\left\{\mathrm{\Psi }\left(\underset{1\le j\le m}{\max }|Z_j|/c\right)\right\}=E\left\{\underset{1\le j\le m}{\max }\mathrm{\Psi }(|Z_j|/c)\right\}\le E\left\{{\displaystyle \sum _{j=1}^{m}E\mathrm{\Psi }(|Z_j|/c)}\right\}\le m·\underset{1\le j\le m}{\max }E\mathrm{\Psi }(|Z_j|/c).$$

Letting c ↘ τ yields

$$E\left\{\underset{1\le j\le m}{\max }|Z_j|\right\}\le \tau {\mathrm{\Psi }}^{-1}\left(m·\underset{1\le j\le m}{\max }\mathrm{\Psi }(|Z_j|/\tau )\right)=\tau {\mathrm{\Psi }}^{-1}(m).$$

Proof of Lemma 7.4

For any u > 0 and v > 0 concavity of ψ⁻¹ implies that

$${\psi }^{-1}(u)+{\psi }^{-1}(v)\ge {\psi }^{-1}(u+v).$$

Therefore, by using this with u = log(1 + m) and v = t, a union bound, and Lemma 7.1,

$$P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \tau ({\psi }^{-1}(\log (1+m))+{\psi }^{-1}(t)))\right)\le P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \tau {\psi }^{-1}(\log (1+m)+t)\right)\le {\displaystyle \sum _{j=1}^{m}P(|Z_j|\ge \tau {\psi }^{-1}(\log (1+m)+t))}\le 2m\exp (-(t+\log (1+m)))=2\frac{m}{m+1}{e}^{-t}\le 2e^{-t}.$$

Proof of Lemma 7.5

By Lemma 7.4

$$P\left(\underset{1\le j\le m}{\max }|Z_j|\ge \tau ({\psi }^{-1}(\log (1+m))+{\psi }^{-1}(t)))\right)\le 2e^{-t}\text{for all }t>0,$$

so the hypothesis of Lemma 7.2 holds for

$$Z\equiv {\left(\underset{1\le j\le m}{\max }|Z_j|-\tau {\psi }^{-1}(\log (1+m))\right)}_{+}$$

with $L=\sqrt{2}$ and τ replaced by $\sqrt{2}\tau$. Thus the conclusion of Lemma 7.2 holds for Z with these choices of L and $\tau :\Vert Z{\Vert }_{\mathrm{\Psi }(·;\sqrt{6})}\le \sqrt{6}\tau$.