Inequalities on the Overshoot beyond a Boundary for Independent Summands with Differing Distributions

Abstract

Let {S_n : n ≥ 0} (S₀ = 0) denote the successive sums of independent non-negative random variates, of possibly differing distributions. Define: (1) the number N(b) = inf{n ≥ 0 : S_n > b} of sums in the interval [0,b]; and (2) the overshoot R_b = S_N(b) −b. This paper bounds the tail ℙ{R_b > c} and the moments $𝔼R_b^k$.

1 Introduction and Statement of Results

As usual, define ℕ₀ := ℕ∪{0} := {0,1,2,…} (“:=” represents a definition). Let (X_n ≥ 0 : n ∈ ℕ) be a sequence of independent non-negative random variates, not necessarily identically distributed. Denote sums of the {X_n} by $S_n≔{\Sigma }_{i=0}^n{X}_i$, where S₀ := X₀ := 0, and assume that lim_n→∞ S_n = ∞ with probability 1. For any set A ⊆ ℕ, define the number N_mA := #{n ≥ m : S_n ∈ A} of sums following epoch m and falling in the set A. In particular, define

$$N_m\left(b\right)≔N_m[0,b]=\inf \{n\ge m:S_n>b\},$$ (1)

where inf Ø := ∞. For brevity, we sometimes drop the subscript m = 0 on N(b) := N₀(b). This paper presents inequalities on the distribution of the overshoot R_b := S_N(b) −b beyond the boundary b.

The overshoot plays a central role in renewal theory (Asmussen 2003), where the {X_n} are usually taken to be independent and identically distributed. In that case, let X =_d X₁ be the generic distributional representative of {X_n}. Elegant coupling methods (Lindvall 1992) yield inequalities on the tail sup_b≥0 ℙ{R_b > c} and the moments ${\sup }_{b\ge 0}𝔼R_b^k$. The best bounds known are $𝔼R_b^k\le {(k+1)}^{-1}(k+2)𝔼X^{k+1}∕𝔼X$ (Chang 1994), with the improvement known as Lorden's inequality 𝔼R_b ≤ 𝔼X²/𝔼X for k = 1 (Lorden 1970). The coupling proofs rely heavily on the existence of a stationary distribution lim_b→∞ ℙ{R_b > c} for the overshoot, however, so ready generalizations to non-identically distributed {X_n} seem unavailable.

Lorden (1970) mentions the bound sup_b≥0 ℙ{R_b > c} ≤ sup_x≥0 ℙ{X > x + c|X > x}, where the right side is 0 if ℙ{X > x} = 0. Without giving a specific page number, he ascribes the bound to Wald (1947). Wald's bound becomes particularly simple, if the failure rate (d/dx)[−log ℙ{X > x}] of X is monotone increasing for x ≥ 0 : because then log ℙ{X > x} is concave, i.e., ℙ{X > x + c|X > x} ≤ ℙ{X > c|X > 0}, so sup_b≥0 ℙ{R_b > c} ≤ ℙ{X > c|X > 0}, with equality if X is exponential. Brown (1980) also gives some related results.

Inequalities on the tail sup_b≥0 ℙ{R_b > c} are useful outside classical renewal theory. Nonlinear renewal theory (Lai and Siegmund 1977; Lai and Siegmund 1979), where the {X_n} might be neither independent nor identically distributed, sometimes requires the hypothesis lim_c→∞ sup_b≥0 ℙ{R_b > c} = 0 (Woodroofe 1990). Other applications, such as the statistical theory justifying the bootstrapping a database ((Green and Brenner 2002); see also the Appendix of (Schaffer et al. 2001)), require inequalities on sup_b≥0 ℙ{R_b > c} for {X_n} independent but not identically distributed. Accordingly, this paper presents overshoot inequalities for the case of independent but non-identically distributed {X_n}. With extra work, our methods can be applied in even more general circumstances.

To state our main result, let X ≤_d X′ denote stochastic inequality: ℙ{X ≥ x} ≤ ℙ{X′ ≥ x} for all x ∈ ℕ and let X =_d X′ denote equality of distribution for X and X′. Assume Y̮_m ≤_d X_n ≤_d Y̑_m for all n > m ≥ 0, where (Y̮_n ≥ 0: n ∈ ℕ₀) is stochastically increasing (i.e., Y̮_n ≤_d Y̮_n+1); and (Y̑_n ≥ 0: n ∈ ℕ₀), stochastically decreasing. For brevity, let Y̮ := Y̮₀ and Y̑ := Y̑₀, and assume that 0 < 𝔼Y̮ ≥ 𝔼Y̑ < ∞.

We note in passing that the bound Lorden attributes to Wald generalizes directly: for any b,c ≥ 0 and m ∈ ℕ₀,

$$\mathbb{P}\{R_b>c\}\le \mathbb{P}\{S_m>b+c\}+{\sup }_{y\ge 0}\frac{\mathbb{P}\left\{{\stackrel{⌢}{Y}}_m>y+c\right\}}{\mathbb{P}\left\{{\underset{⌣}{Y}}_m>y\right\}},$$ (2)

where the right side is 0 if ℙ {Y̮_m > y}=0. We prove Eq (2) later, but our main result is Theorem 1.1 below.

Let ${\underset{⌣}{N}}_{m}A≔\#\left\{n\ge 0:{\underset{⌣}{S}}_{m,n}\in A\right\}$, where ${\underset{⌣}{S}}_{m,n}≔{\Sigma }_{i=0}^n{\underset{⌣}{Y}}_{m,i},{\underset{⌣}{Y}}_{m,0}≔0$, and the (${\underset{⌣}{Y}}_{m,i}:i\in \mathbb{N}$) are independent variates with the same distribution as Y̮_m. Abbreviate the average density of renewals as ${\underset{⌣}{\rho }}_m\left(\delta \right)≔{\delta }^{-1}𝔼{\underset{⌣}{N}}_m[0,\delta )$, and $\underset{⌣}{\rho }\left(\delta \right)≔{\underset{⌣}{\rho }}_0\left(\delta \right)$, and use the unambiguous abbreviation $x^{n+}≔{\left(x^{+}\right)}^n$, where $x^{+}≔\max \{0,x\}$.

Theorem 1.1

Under the above set-up, for any b,c ≥ 0, δ > 0, and m ∈ ℕ₀,

$$\mathbb{P}\{R_b>c\}\le \mathbb{P}\{S_m>b+c\}+𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta -c\right]}^{+}{\underset{⌣}{\rho }}_m\left(\delta \right).$$ (3)

. From $𝔼R_b^k={\int }_0^{\infty }{\mathit{kc}}^{k-1}\mathbb{P}\left\{R_b>c\right\}\mathit{dc}$ and ${(k+1)}^{-1}𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta \right]}^{k+1}={\int }_0^{\infty }{\mathit{kc}}^{k-1}𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta -c\right]}^{+}\mathit{dc}$, the integration of Eq (3) over c ∈ [0, ∞) yields

$$𝔼R_b^k\le 𝔼{\left[S_m-b\right]}^{k+}+{(k+1)}^{-1}𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta \right]}^{k+1}{\underset{⌣}{\rho }}_m\left(\delta \right).$$ (4)

Note that the bounds in Eq (3) and (4) can be minimized over δ > 0 and m ∈ ℕ₀.

Because 0 ≤ [Y̑ + δ − c]⁺ ≤ Y̑ + δ, we have lim_c→∞ 𝔼[Y̑ + δ − c]⁺ = 0 by dominated convergence. From Eq (3) with m = 0 and any δ > 0 , the conditions of Theorem 1.1 suffice (Woodroofe 1990) to demonstrate that lim_c→∞ sup_b≥0 ℙ{R_b > c} = 0.

With m = 0, Eqs (3) and (4) show that ℙ{R_b > c} ≤ 𝔼 [Y̑ + δ−c]⁺ ρ̮(δ) and $𝔼R_b^k\le {(k+1)}^{-1}𝔼{\left[\stackrel{⌢}{Y}+\delta \right]}^{k+1}\underset{⌣}{\rho }\left(\delta \right)$ independently of b ≥ 0. The bounds are particularly useful in theoretical contexts, because the best bounds known for identically distributed random variates {X_n} (Chang 1994) contain moments of the same order.

If required, the usual methods yield explicit bounds for the renewal density ρ̮_m(δ). Coupling, e.g., shows that if p_m (δ) := ℙ{Y̮_m ≥ δ}, then Bernoulli (p_m(δ)) ≤_d δ⁻¹Y̮_m, so N̮_m [0,δ)≤_d Geometric (p_m(δ)) (the right side being the waiting time for the sum of independent Bernoulli (p_m(δ)) variates to leave [0,1)). Thus, 𝔼N̮_m [0,δ)≤1/ p_m (δ) so ρ̮_m (δ)≥ δ⁻¹ / p_m (δ). Thus, from Eq (4) for m = 0 and k = 1,

$$\underset{b\ge 0}{\sup }𝔼R_b\le \frac{1}{2}\underset{\delta >0}{\inf }\frac{𝔼{\left[\stackrel{⌢}{Y}+\delta \right]}^2}{\delta \mathbb{P}\left\{\stackrel{⌢}{Y}\ge \delta \right\}}.$$ (5)

Although Eq (5) does not reduce to equality in any special case, it compares reasonably well with Lorden's inequality sup_b≥0 𝔼R_b ≤ 𝔼X²/𝔼X, which hypothesizes variates with equal distributions.

In passing, note that the moment generating function ϕ_m (θ) := 𝔼 exp (θY̮_m) and a Chernoff-type inequality with θ < 0 also yield a bound for the renewal density ρ̮_m (δ) :

$$\begin{array}{cc}\hfill 𝔼{\underset{⌣}{N}}_m[0,\delta )& =\underset{n=0}{\overset{\infty }{\Sigma }}\mathbb{P}\left\{{\underset{⌣}{S}}_{m,n}<\delta \right\}\le \underset{n=0}{\overset{\infty }{\Sigma }}𝔼\left[\exp \left\{\theta \left({\underset{⌣}{S}}_{m,n}-\delta \right)\right\};{\underset{⌣}{S}}_{m,n}<\delta \right]\hfill \\ \hfill & \le \underset{n=0}{\overset{\infty }{\Sigma }}𝔼\left[\exp \left\{\theta \left({\underset{⌣}{S}}_{m,n}-\delta \right)\right\}\right]\le e^{-\theta \delta }\underset{n=0}{\overset{\infty }{\Sigma }}{\varphi }_m^n\left(\theta \right)=e^{-\theta \delta }∕\left\{1-{\varphi }_m\left(\theta \right)\right\}.\hfill \end{array}$$ (6)

.

The next section proves Theorem 1.1 and concludes with a brief proof of the inequality Eq (2).

2 Proof of Theorem 1.1

Let us bound the tail of the overshoot distribution:

$$\begin{array}{cc}\hfill \mathbb{P}\left\{R_b>c\right\}& =\underset{[0,b]}{\int }\underset{n\ge 1}{\Sigma }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy};X_n>y+c\right\}\hfill \\ \hfill & \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\underset{n>m}{\Sigma }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy};X_n>y+c\right\}\hfill \\ \hfill & =\mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\underset{n>m}{\Sigma }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy}\right\}\mathbb{P}\left\{X_n>y+c\right\}\hfill \\ \hfill & \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\underset{n>m}{\sup }\mathbb{P}\left\{X_n>y+c\right\}\underset{n\ge m}{\Sigma }\mathbb{P}\left\{S_n=b-\mathit{dy}\right\}\hfill \\ \hfill & \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\mathbb{P}\left\{{\stackrel{⌢}{Y}}_m>y+c\right\}𝔼N_m\{b-\mathit{dy}\}.\hfill \end{array}$$ (7)

.

To continue from Eq (7), let j = ⌊b/δ⌋, so $[0,b]\subset {\bigcup }_{i=0}^j[i\delta ,(i+1)\delta )$. Substitute the inclusion into the domain of integration, noting that ℙ{Y̑_m > y + c} ≤ ℙ{Y̑_m > iδ + c} for y ∈ [iδ,(i + 1)δ) :

$$\begin{array}{cc}\hfill \mathbb{P}\left\{R_b>c\right\}& \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{i=0}{\overset{j}{\Sigma }}\mathbb{P}\left\{{\stackrel{⌢}{Y}}_m>i\delta +c\right\}𝔼N_m(b-(i+1)\delta ,b-i\delta ]\hfill \\ \hfill & \le \mathbb{P}\left\{S_m>b+c\right\}+𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta -c\right]}^{+}{\sup }_{t\in ℝ}{\delta }^{-1}𝔼N_m(t,t+\delta ],\hfill \end{array}$$ (8)

because ℙ{Y̑_m > iδ + c} ≤ ℙ{Y̑_m > y − δ + c} for y ∈ [iδ,(i + 1)δ), and

$$\delta \underset{i=0}{\overset{\infty }{\Sigma }}\mathbb{P}\left\{{\stackrel{⌢}{Y}}_m>i\delta +c\right\}\le \underset{y=0}{\overset{\infty }{\int }}\mathbb{P}\left\{{\stackrel{⌢}{Y}}_m>y-\delta +c\right\}\mathit{dy}=𝔼{\left[{\stackrel{⌢}{Y}}_m+\delta -c\right]}^{+}.$$ (9)

.

Eq (8) motivates a bound on 𝔼N̮_m (t,t+δ]. A minor variation on a standard coupling argument (e.g., (Lindvall 1992)) provides the bound 𝔼N̮_m (t,t+δ] ≤ 𝔼N̮_m [0,δ). To sketch the argument, construct a probability space containing sequences $\{{\underset{⌣}{Y}}_{m,n}^{\prime }:n\in {\mathbb{N}}_0\}$ and $\{{\underset{⌣}{Y}}_{m,n}^{″}:n\in {\mathbb{N}}_0\}$, where ${\underset{⌣}{Y}}_{m,n}^{\prime }{=}_d{\underset{⌣}{Y}}_{m,n}^{″}{=}_d{\underset{⌣}{Y}}_{m,n}$ for $n\in {\mathbb{N}}_0$ and ${\underset{⌣}{Y}}_{m,N\left(t\right)+n}^{\prime }={\underset{⌣}{Y}}_{m,n}^{″}$ for $n\in \mathbb{N}$. In an obvious notation, ${\underset{⌣}{N}}_m^{\prime }(t,t+\delta ]\le {\underset{⌣}{N}}_m^{″}[0,\delta )$ on this space, so Eq (3) follows. This concludes the proof of Theorem 1.1.

A brief demonstration of the inequality Eq (2) follows. Start as in Eq (7):

$$\begin{array}{cc}\hfill \mathbb{P}\left\{R_b>c\right\}& \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\underset{n>m}{\Sigma }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy}\right\}\mathbb{P}\left\{X_n>y+c\right\}\hfill \\ \hfill & =\mathbb{P}\left\{S_m>b+c\right\}+\underset{[0,b]}{\int }\underset{n>m}{\Sigma }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy};X_n>y\right\}\mathbb{P}\left\{X_n>y+c\mid X_n>y\right\}\hfill \\ \hfill & \le \mathbb{P}\left\{S_m>b+c\right\}+\underset{n>m,y\ge 0}{\sup }\mathbb{P}\left\{X_n>y+c\mid X_n>y\right\}\underset{n>m}{\Sigma }\underset{[0,b]}{\int }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy};X_n>y\right\}.\hfill \end{array}$$ (10)

Because

$$\underset{n>m}{\Sigma }\underset{[0,b]}{\int }\mathbb{P}\left\{S_{n-1}=b-\mathit{dy};Y_n>y\right\}=\underset{n>m}{\Sigma }\mathbb{P}\{N\left(b\right)=n\}\le 1,$$ (11)

Eq (2) is proved.