Asymptotics of nonparametric L-1 regression models with dependent data

Abstract

We investigate asymptotic properties of least-absolute-deviation or median quantile estimates of the location and scale functions in nonparametric regression models with dependent data from multiple subjects. Under a general dependence structure that allows for longitudinal data and some spatially correlated data, we establish uniform Bahadur representations for the proposed median quantile estimates. The obtained Bahadur representations provide deep insights into the asymptotic behavior of the estimates. Our main theoretical development is based on studying the modulus of continuity of kernel weighted empirical process through a coupling argument. Progesterone data is used for an illustration.

1. Introduction

There is a vast literature on the nonparametric location-scale model Y = μ(X) + s(X)e, where X, Y, and e are the covariates, response, and error, respectively. Given observations {(X_j, Y_j)}_j=1,…,m, the latter model has been studied under various settings of data structure. In terms of the dependence structure, there are independent data and time series data scenarios; in terms of the design point X, there are random-design and fixed-design X_j = j/m settings. In these settings, we usually assume that either (X_j, Y_j) are independent observations from subjects j = 1, …, m, or {(X_j, Y_j)}_{j=1,…, m} is a sequence of time series observations from the same subject. We refer the reader to Fan and Yao (2003) and Li and Racine (2007) for an extensive exposition of related works.

In this article we are interested in the following nonparametric location-scale model with serially correlated data from multiple subjects:

$$Y_{i,j}=\mu (x_{i,j})+s(x_{i,j})e_{i,j},1\le j\le m_i,1\le i\le n,$$ (1.1)

where, for each subject i, {(x_i,j, Y_i,j)}_j=1,…,mi is the sequence of covariates and responses, and {e_i,j}_j=1,…,mi is the corresponding error process. We study (1.1) under a general dependence framework for {e_i,j}_j∈ℕ that allows for both longitudinal data and some spatially correlated data. In typical longitudinal studies, x_i,j represents measurement time or covariates at time j, then it is reasonable to assume that {e_i,j}_j∈ is a causal time series, i.e., the current observation depends only on past but not future observations. In other applications, however, measurements may be dependent on both the left and right neighboring measurements, especially when x_i,j represents measurement location. A good example of this type of data is the vertical density profile data in Walker and Wright (2002); see also Section 2.1 for more details. To accommodate this, we propose a general error dependence structure, which can be viewed as an extension of the one-sided causal structure in Wu (2005) and Dedecker and Prieur (2005) to a two-sided non-causal setting. The proposed dependence framework allows for many linear and nonlinear processes.

We are interested in nonparametric estimation of the location function μ(·) and the scale function s(·). Least-squares based nonparametric methods have been extensively studied for both time series data (Fan and Yao, 2003) and longitudinal data (Hoover et al., 1998; Fan and Zhang, 2000; Wu and Zhang, 2002; Yao, Müller and Wang, 2005). While they perform well for Gaussian errors, least-squares based methods are sensitive to extreme outliers, especially when the errors have a heavy-tailed distribution. By contrast, robust estimation methods impose heavier penalty on far-deviated data points to reduce the impact from extreme outliers. For example, median quantile regression uses the absolute loss and the resultant estimator is based on sample local median. Since Koenker and Bassett (1978), quantile regression has become popular in parametric and nonparametric inferences and we refer the reader to Yu, Lu and Stander (1998) and Koenker (2005) for excellent expositions. Recently, He, Fu and Fung (2003), Koenker (2004) and Wang and Fygenson (2009) applied quantile regression techniques to parameter estimation of parametric longitudinal models, He, Zhu and Fung (2002) studied median regression for semiparametric longitudinal models, and Wang, Zhu and Zhou (2009) studied inferences for a partially linear varying-coefficient longitudinal model. Here we focus on quantile regression based estimation for the nonparametric model (1.1).

We aim to study the asymptotic properties, including uniform Bahadur representations and asymptotic normalities, of the least-absolute-deviation or median quantile estimates for model (1.1) under a general dependence structure. Nonparametric quantile regression estimation has been studied mainly under either the iid setting (Bhattacharya and Gangopadhyay, 1990; Chaudhuri, 1991; Yu and Jones, 1998) or the strong mixing setting (Truong and Stone, 1992; Honda, 2000; Cai, 2002). There are relatively scarce results on Bahadur representations of conditional quantile estimates. Bhattacharya and Gangopadhyay (1990) and Chaudhuri (1991) obtained point-wise Bahadur representations for conditional quantile estimation of iid data. For mixing stationary processes, Honda (2000) obtained point-wise and uniform Bahadur representations of conditional quantile estimates. For stationary random fields, Hallin, Lu and Yu (2009) obtained a point-wise Bahadur representation for spatial quantile regression function under spatial mixing conditions. Due to the non-stationarity and dependence structure, it is clearly challenging to establish Bahadur representations in the context of (1.1).

Our contribution here is mainly on the theoretical side. We establish uniform Bahadur representations for the least-absolute-deviation estimates of μ(·) and σ(·) in (1.1). To derive the uniform Bahadur representations, the key ingredient is to study the modulus of continuity of certain kernel weighted empirical processes of the non-stationary observations Y_i,j in (1.1). Empirical processes have been extensively studied under various settings, including the iid setting (Shorack and Wellner, 1986), linear processes (Ho and Hsing, 1996), strong mixing setting (Andrews and Pollard, 1994; Shao and Yu, 1996), and general causal stationary processes (Wu, 2008). Using a coupling argument to approximate the dependent process by an m-dependent process with a diverging m, we study the modulus of continuity of weighted empirical processes, and the latter result serves as a key tool in establishing our uniform Bahadur representations. These Bahadur representations provide deep insights into the asymptotic behavior of the estimates, and in particular they provide theoretical justification for the profile control chart methodologies in Wei, Zhao and Lin (2012). These technical treatments are also of interest in other nonparametric problems involving dependent data.

The article is organized as follows. In Section 2 we introduce the error dependence structure with examples. In Section 3 we study weighted empirical process through a coupling argument. Section 4 contains uniform Bahadur representations and asymptotic normality. Section 5 contains an illustration using progesterone data. Possible extensions to spatial setting are discussed in Section 6. Proofs are provided in Section 7.

2. Error dependence structure

First we introduce some notation used throughout this article. For a, b ∈ ℝ, let ⌊a⌋ be the integer part of a, a ∨ b = max(a, b), and a ∧ b = min(a, b). For a random variable Z ∈ , q > 0, if ||Z||_q = [ (|Z|^q)]^1/q < ∞. Let ( ) be the set of functions with bounded derivatives up to order r on a set ⊂ ℝ.

Assume that, for each i, the error process {e_i,j}_j∈ℕ in (1.1) is an independent copy from a stationary process {e_j}_j∈ℕ which has the representation

$$e_j=G({\epsilon }_j,{\epsilon }_{j\pm 1},{\epsilon }_{j\pm 2},\dots ),$$ (2.1)

where ε_j, j ∈ , are iid random innovations, and G is a measurable function such that e_j is well defined. We can view (2.1) as an input-output system with (ε_j, ε_j±1, ε_j±2, …), G, and e_j being, respectively, the input, filter, and output. Wu (2005) considered the causal time series case that e_j depends only on the past innovations ε_j, ε_j−1, …. In contrast, (2.1) allows for non-causal models and is particularly useful for applications that do not have a time structure. For example, if x_i,j are locations, then the corresponding measurement y_i,j depends on both the left and right neighboring measurements.

Condition 2.1

Let ${\{{\epsilon }_j^{\prime }\}}_{j\in \mathbb{Z}}$ be iid copies of {ε_j}_j∈ . There exist constants q > 0 and ρ ∈ (0, 1) such that

$${\Vert e_0-e_0(k)\Vert }_q=O({\rho }^k),\text{where}{e}_0(k)=G({\epsilon }_0,{\epsilon }_{\pm 1},\dots ,{\epsilon }_{\pm k},{\epsilon }_{\pm (k+1)}^{\prime },{\epsilon }_{\pm (k+2)}^{\prime },\dots ).$$ (2.2)

In (2.2), e₀(k) can be viewed as a coupling process of e₀ with {ε_r}_|r|≥k+1 replaced by the iid copy ${\{{\epsilon }_r^{\prime }\}}_{\mid r\mid \ge k+1}$ while keeping the nearest 2k +1 innovations {ε_r}_|r|≤k. In particular, if e₀ does not depend on {ε_r}_|r|≥k+1, then e₀(k) = e₀. Thus, ||e₀ − e₀(k)||_q quantifies the contribution of {ε_r}_|r|≥k+1 to e₀, and (2.2) states that the contribution decays exponentially in k. Shao and Wu (2007) and Dedecker and Prieur (2005) [cf. Equation (4.2) therein] considered one-sided causal version of (2.2) where e₀ depends only on {ε_r}_r≤0.

Propositions 2.1–2.2 below indicate that, if {e_i} satisfies (2.2), then its properly transformed process also satisfies (2.2).

Proposition 2.1

For 0 < ς ≤ 1 and υ ≥ 0, define the collection of functions h

$$\mathcal{H}(\varsigma ,\upsilon )=\{h:\mid h(x)-h(x^{\prime })\mid \le c{\mid x-x^{\prime }\mid }^{\varsigma }{(1+\mid x\mid +\mid x^{\prime }\mid )}^{\upsilon },x,x^{\prime }\in \mathbb{R}\},$$ (2.3)

where c is a constant. Suppose {e_j} satisfies (2.2). Then the transformed process $e_j^{\ast }=h(e_j)$ satisfies (2.2) with (q, ρ) replaced by q^* = q/(ς + υ) and ρ^* = ρ^ς.

In (2.3), (ς, 0) is the class of uniformly Hölder-continuous functions with index ς. If h(x) = |x|^b, b > 1, then h ∈ (1, b − 1). Clearly, all functions in (ς, 0) are continuous. Interestingly, for non-continuous transformations, the conclusion may still hold; see Proposition 2.2 below, where 1 is the indicator function.

Proposition 2.2

Let e₀ have a bounded density. Suppose {e_j} satisfies (2.2). Then, for any given x, {1 _ej≤x} satisfies (2.2) with ρ replaced by ρ^* = ρ^1/(1+q).

Propositions 2.1–2.2 along with the examples below show that the error structure (2.1) and Condition 2.1 are sufficiently general to accommodate many popular linear and nonlinear time series models and their properly transformed processes.

Example 2.1 (m-dependent sequence)

Assume that e_j = G(ε_j, ε_j±1, …, ε_j±m) for a measurable function G. Then e_j depends only on the nearest 2m + 1 innovations ε_j, ε_j±1, …, ε_j±m. Clearly, {e_j}_j∈ form a (2m+1)-dependent sequence, ||e₀−e₀(k)||_q = 0 for k ≥ m, and (2.2) trivially holds. If m = 0, then e_j are iid random variables.

Example 2.2 (Non-causal linear processes)

Consider the non-causal linear process $e_j={\sum }_{r=-\infty }^{\infty }{a}_r{\epsilon }_{j-r}$. If ε_j ∈ and a_j = O(ρ^|j|), then it is easy to see that (2.2) holds.

Example 2.3 (Iterated random functions)

Consider random variables e_j defined by

$$e_j=R(e_{j-1},\dots ,e_{j-d};{\epsilon }_j),$$ (2.4)

where ε_j, j ∈ , are iid random innovations, and R is a random map. Many widely time series models are of form (2.4), including threshold autoregressive model e_j = a max(e_j−1, 0) + b min(e_j−1, 0) + ε _j, autoregressive conditional heteroscedastic model $e_j={\epsilon }_j{(a^2+b^2{e}_{j-1}^2)}^{1/2}$, random coefficient model e_j = (a + bε_j)e_j−1 + ε_j, and exponential autoregressive model $e_j=[a+bexp(-{ce}_{j-1}^2)]e_{j-1}+{\epsilon }_j$, among others. Suppose there exists z₀ such that R(z₀; ε₀) ∈ and there exist constants a₁, …, a_d such that

$$\sum _{j=1}^d{a}_j<1\text{and}{\Vert R(z;{\epsilon }_0)-R(z^{\prime };{\epsilon }_0)\Vert }_q^{1\wedge q}\le \sum _{j=1}^d{a}_j{\mid z_j-z_j^{\prime }\mid }^{1\wedge q}$$

holds for all z = (z₁, …, z_d), $z^{\prime }=(z_1^{\prime },\dots ,z_d^{\prime })$. By Shao and Wu (2007), (2.2) holds.

2.1. Some examples

The imposed dependence structure and hence the developed results in Sections 3–4 below are potentially applicable to a wide range of practical data types. We briefly mention some below.

(Time series data)

In the special case of n = 1, m₁ = m → ∞ and (x_1,j, Y_1,j, e_1,j) = (x_j, Y_j, e_j) for a stationary time series {e_j}, (1.1) becomes the usual nonparametric location-scale model Y_j = μ(x_j) + s(x_j)e_j with time series data. The latter model has been extensively studied under both the random-design case and the fixed-design case x_j = j/n. See Fan and Yao (2003) for an excellent introduction to various local least-squares based methods under mixing settings. Quantile regression based estimations have been studied in Truong and Stone (1992), Honda (2000), and Cai (2002) for mixing processes. Despite the popularity of mixing conditions, it is generally difficult to verify mixing conditions even for linear processes. For example, for the autoregressive model X_i = ρX_i−1 + ε_i, ρ ∈ (0, 1/2], where ε_i are iid Bernoulli random variables ℙ(ε_i = 1) = 1 − ℙ(ε_i = 0) = q ∈ (0, 1), the stationary solution is not strong mixing (Andrews, 1984). By contrast, as shown above, the imposed Condition 2.1 is easily verifiable for many linear and nonlinear time series models and their proper transformations.

Longitudinal data

For each subject i, if x_i,j is the j-th measurement time or the covariates at time j, Y_i,j is the corresponding response, and {e_i,j}_j∈ℕ is a stationary causal process [for example, e_j = G(ε_j, ε_j−1, ε_j−2, …) in (2.1) depends only on the past], then (1.1) becomes a typical longitudinal data setting. For example, Section 5.2 re-examines the well-studied progesterone data using the proposed methods. Another popular longitudinal data example is the CD4 cell percentage in HIV infection from the Multicenter AIDS Cohort Study. Based on least-squares methods, this data has been studied previously in Hoover et al. (1998) and Fan and Zhang (2000). We can examine how the response function (CD4 cell percentage) varies with measurement time (age) using the proposed robust estimation method in Section 4.

Spatially correlated data

In the vertical density data of Walker and Wright (2002), manufacturers are concerned about engineered wood boards’ density, which determines fiberboard’s overall quality. For each board, densities are measured at various locations along a designated vertical line. In this example, measurements depend on both the left and right neighboring measurements, and it is reasonable to impose the dependence structure (2.1). See Wei, Zhao and Lin (2012) for a detailed analysis. Also, as will be discussed in Section 6, the two-sided framework (2.1) can be extended to spatial lattice settings. We point out that the structure in (1.1) and (2.1) differs from the usual spatial model setting in the sense that (1.1) allows for observations from multiple independent subjects whereas the latter usually assumes that all observations are spatially correlated [see, e.g., Hallin, Lu and Yu (2009) for quantile regression of spatial data].

3. Weighted empirical process

In this section, we study weighted empirical processes through a coupling argument. Dependence is the main difficulty in extending results developed for independent data to dependent data. For mixing processes, the widely used large-block-small-block technique partitions the data into asymptotically independent blocks. Here, we adopt a coupling argument which copes well with the dependence structure in Section 2.

We now illustrate the basic idea. By (2.1), the error e_i,j in (1.1) has the representation

$$e_{i,j}=G({\epsilon }_{i,j},{\epsilon }_{i,j\pm 1},{\epsilon }_{i,j\pm 2},\dots ),$$

for iid innovations ε_i,j, i, j ∈ . Thus, {e_i,j}_j∈ is a dependent series for each fixed i, whereas {e_i1,j}_j∈ and {e_i2,j}_j∈ are two independent series for i₁ ≠ i₂. Let ${\epsilon }_{i,j,k}^{\prime },i,j,k\in \mathbb{Z}$, be iid copies of ε_i,j. For k_n ∈ ℕ, define the coupling process of e_i,j as

$$e_{i,j}(k_n)=G({\epsilon }_{i,j},{\epsilon }_{i,j\pm 1},\dots ,{\epsilon }_{i,j\pm k_n},{\epsilon }_{i,j,j\pm (k_n+1)}^{\prime },{\epsilon }_{i,j,j\pm (k_n+2)}^{\prime },\dots )$$ (3.1)

by replacing all but the nearest 2k_n + 1 innovations with iid copies. We call k_n the coupling lag. Clearly, e_i,j(k_n) has the same distribution as e_i,j.

By construction, for each fixed i, {e_i,j(k_n)}_j∈ form (2k_n + 1)-dependent sequence in the sense that e_i,j(k_n) and e_i,j′(k_n) are independent if |j − j′| ≥ 2k_n + 1. Consequently, for each fixed i and s, {e_{i,(j−1)(2kn+1)+s}(k_n)}_j∈ are iid. The latter property helps us reduce the dependent data to an independent case. On the other hand, under (2.2), ||e_i,j − e_i,j(k_n)||_q = O(ρ^k_n) is sufficiently small with properly chosen k_n and hence the coupling process is close enough to the original one. Similarly, for Y_i,j in (1.1), define its coupling process:

$${\stackrel{\sim }{Y}}_{i,j}=\mu (x_{i,j})+s(x_{i,j})e_{i,j}(k_n).$$ (3.2)

First, we present a general result regarding the sum of functions of the coupling process Ỹ_i,j. Let be any finite set. For real-valued functions g_i,j(y, v), i, j ∈ ℕ, defined on ℝ × such that [g_i,j(Ỹ_i,j, v)] = 0 for all v ∈ , define

$$H_n(v)=\sum _{i=1}^n\sum _{j=1}^{m_i}{g}_{i,j}({\stackrel{\sim }{Y}}_{i,j},v),v\in {\mathcal{V}}_n.$$

Throughout let N_n = m₁ + ··· + m_n be the total number of observations.

Theorem 3.1

Assume that the cardinality | | of and the coupling lag k_n grow no faster than a polynomial of N_n. Further assume |g_i,j(y, v)| ≤ c for a constant c < ∞, and for some sequence χ_n,

$$\underset{v\in {\mathcal{V}}_n}{max}\sum _{i=1}^n\sum _{j=1}^{m_i}\mathbb{E}[g_{i,j}^2({\stackrel{\sim }{Y}}_{i,j},v)]\le {\chi }_n.$$ (3.3)

1. If χ_n = O(1), then max_v∈ |H_n(v)| = O_p(k_n log N_n).

2. If sup_n log N_n/χ_n < ∞, then max_v∈ |H_n(v)| = O_p[k_n(χ_n log N_n)^1/2].

By Theorem 3.1, the magnitude of max_v∈ |H_n(v)| increases with the coupling lag k_n. Intuitively, as k_n increases, there is stronger dependence in the coupling process Ỹ_i,j and consequently a larger bound for H_n(v). Therefore, a small k_n is preferred in order to have a tight bound for H_n(v). On the other hand, a reasonably large k_n is needed in order for the coupling process to be sufficiently close to the original process. Under (2.2), for k_n = O(log N_n), the coupling process converges to the original one at a polynomial rate, and meanwhile the maximum bound in Theorem 3.1 is optimal up to a logarithm factor. For example, if χ_n = O(1), then max_v∈ |H_n(v)| = O_p[(log N_n)²]; if sup_n log N_n/χ_n < ∞, then max_v∈ |H_n(v)| = O_p{[χ_n(log N_n)³]^1/2}.

In what follows we consider the special case of weighted empirical process, which plays an essential role in quantile regression. Let ϖ_i,j(x) ≥ 0 be non-random weights that may depend on x. Consider the weighted empirical process

$$F_n(x,y)=\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}(x){\mathbf{1}}_{Y_{i,j}\le y}.$$ (3.4)

To study F_n(x, y), recall Ỹ_i,j in (3.2) and define the coupling empirical process

$${\stackrel{\sim }{F}}_n(x,y)=\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}(x){\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le y}.$$ (3.5)

Under mild regularity conditions, Theorem 3.2 below states that F_n(x, y) can be uniformly approximated by F̃_n(x, y) with proper choice of the coupling lag k_n.

Condition 3.1

(i) ϖ_i,j(x) ≤ c uniformly for some constant c < ∞. (ii) μ(x_i,j) is uniformly bounded. (iii) s(x_i,j) > 0 is uniformly bounded away from zero and infinity.

Theorem 3.2

Assume that Conditions 2.1 and 3.1 hold. In (3.1), let the coupling lag k_n = ⌊λlog N_n⌋ for some λ > (q + 1)/[q log(1/ρ)], where N_n = m₁ + ··· + m_n. Then

$$\underset{x,y\in \mathbb{R}}{sup}\mid F_n(x,y)-{\stackrel{\sim }{F}}_n(x,y)\mid =O_{\text{p}}[{(log{N}_n)}^2].$$

To study asymptotic Bahadur representations of quantile regression estimates, a key step is to study the modulus of continuity of F_n(x, y), defined by

$$D_n(\delta ,x,y)=\{F_n(x,y+\delta )-\mathbb{E}[F_n(x,y+\delta )]\}-\{F_n(x,y)-\mathbb{E}[F_n(x,y)]\}.$$ (3.6)

Intuitively, D_n(δ, x, y) measures the oscillation of the centered empirical process F_n(x, y)– [F_n(x, y)] in response to a small perturbation δ in y.

The dependence structure in Section 2 along with the coupling argument provides a convenient framework to study D_n(δ, x, y). Recall F̃_n(x, y) in (3.5). For D_n(δ, x, y) in (3.6), define its coupling process

$${\stackrel{\sim }{D}}_n(\delta ,x,y)=\{{\stackrel{\sim }{F}}_n(x,y,+\delta )-\mathbb{E}[{\stackrel{\sim }{F}}_n(x,y+\delta )]\}-\{{\stackrel{\sim }{F}}_n(x,y)-\mathbb{E}[{\stackrel{\sim }{F}}_n(x,y)]\}.$$ (3.7)

Notice that e_i,j(k_n) and e_i,j have the same distribution, so [F_n(x, y)] = [F̃_n(x, y)]. By Theorem 3.2, it is easy to see that, uniformly over x, y, δ,

$$\mid D_n(\delta ,x,y)-{\stackrel{\sim }{D}}_n(\delta ,x,y)\mid \le 2\underset{x,y\in \mathbb{R}}{sup}\mid F_n(x,y)-{\stackrel{\sim }{F}}_n(x,y)\mid =O_{\text{p}}[{(log{N}_n)}^2].$$ (3.8)

Therefore, the asymptotic properties of D_n(δ, x, y) are similar to that of D̃_n(δ, x, y), which can be studied through Theorem 3.1.

Condition 3.2

(i) ϖ_i,j(·) = 0 outside a common bounded interval for all i, j. (ii) There exist τ_n and φ_n such that

$$\underset{x\ne x^{\prime }}{sup}\frac{\mid {\varpi }_{i,j}(x)-{\varpi }_{i,j}(x^{\prime })\mid }{\mid x-x^{\prime }\mid }\le {\tau }_n\mathit{and}\underset{x}{sup}\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}^2(x)\le {\phi }_n.$$ (3.9)

Theorem 3.3

Assume that Conditions 2.1 and 3.1–3.2 hold. Further assume δ_n → 0, sup_n log N_n/(δ_nφ_n) < ∞, and that 1/δ_n + τ_n grows no faster than a polynomial of N_n. Then

$$\underset{\mid \delta \mid \le {\delta }_n,x,y\in \mathbb{R}}{sup}\mid D_n(\delta ,x,y)\mid =O_{\text{p}}\{{[{\delta }_n{\phi }_n{(log{N}_n)}^3]}^{1/2}\}.$$ (3.10)

4. Quantile regression and Bahadur representations

For a random variable Z, denote by (Z) = inf{z ∈ ℝ, ℙ(Z ≤ z) ≥ 1/2} the median of Z, and similarly denote by (·|·) the conditional median operator. To ensure identifiability of μ and s in (1.1), without loss of generality we assume (e_i,j) = 0 and (|e_i,j|) = 1.

Note that (Y_i,j|x_i,j = x) = μ(x). Applying a kernel localization technique, we propose the following least-absolute-deviation or median quantile estimate of μ(x):

$$\widehat{\mu }(x)=\underset{\theta }{\text{argmin}}\sum _{i=1}^n\sum _{j=1}^{m_i}\mid Y_{i,j}-\theta \mid K_{b_n}(x_{i,j}-x),\text{where}{K}_{b_n}(u)=K(u/b_n)$$ (4.1)

for a non-negative kernel function K satisfying ∫_ℝ K(u) = 1, and b_n > 0 is a bandwidth. The estimate μ̂_bn(x) pools together information across all subjects, an appealing property especially when some subjects have sparse observations. By the Bahadur representation in Theorem 4.1 below, the bias term of μ̂(x)−μ(x) is of order $O(b_n^2)$. Following Wu and Zhao (2007), we adopt a jackknife bias-correction technique. In (4.1), denote by μ̂(x|b_n) and $\widehat{\mu }(x\mid \sqrt{2}{b}_n)$ the estimates of μ(x) using bandwidth b_n and $\sqrt{2}{b}_n$, respectively. The bias-corrected jackknife estimator is

$$\stackrel{\sim }{\mu }(x)=2\widehat{\mu }(x\mid b_n)-\widehat{\mu }(x\mid \sqrt{2}{b}_n),$$ (4.2)

which can remove the second-order bias term $O(b_n^2)$ in μ̂(x).

After estimating μ(·), we estimate s(·) based on residuals. Notice that (|e_i,j|) = 1 implies (|Y_i,j − μ(x)||x_i,j = x) = s(x). Therefore, we propose

$$\widehat{s}(x)=\underset{\theta }{\text{argmin}}\sum _{i=1}^n\sum _{j=1}^{m_i}\left|\mid Y_{i,j}-\stackrel{\sim }{\mu }(x)\mid -\theta \right|K_{h_n}(x_{i,j}-x),$$ (4.3)

where h_n > 0 is another bandwidth, and μ̃(x) is the bias-corrected jackknife estimator in (4.2). As in (4.2), we adopt the following bias-corrected jackknife estimator

$$\stackrel{\sim }{s}(x)=2\widehat{s}(x\mid h_n)-\widehat{s}(x\mid \sqrt{2}{h}_n).$$ (4.4)

Remark 4.1

By (|Y_i,j − μ(x_i,j)||x_i,j = x) = s(x), an alternative estimator of s(x) is

$$\overline{s}(x)=\underset{\theta }{\text{argmin}}\sum _{i=1}^n\sum _{j=1}^{m_i}\left|\mid Y_{i,j}-\stackrel{\sim }{\mu }(x_{i,j})\mid -\theta \right|K_{h_n}(x_{i,j}-x).$$ (4.5)

The difference between (4.3) and (4.5) is that (4.3) uses μ̃(x) whereas (4.5) uses μ̃(x_i,j). Since K has bounded support, only those x_i,j’s with |x_i,j − x| = O(h_n) contribute to the summation in (4.5). Thus, as h_n → 0 so that x_i,j → x and μ̃(x_i,j) ≈ μ̃(x), the two estimators in (4.3) and (4.5) are expected to be asymptotically close. Our use of (4.3) has some technical and computational advantages. First, the estimation error μ̃(x_i,j)−μ(x_i,j) varies with (i, j), and thus it is technically more challenging to study (4.5). Second, to implement (4.5), we need to compute μ̃(·) at each point x_i,j, which requires solving a large number of optimization problems in (4.1) for a large data set. By contrast, (4.3) only requires estimation of μ̃(·) at those grid points x at which we wish to estimate s(·).

To study asymptotic properties, we need to introduce some regularity conditions. Throughout we write ([a, b]) = [a + ε, b − ε] for an arbitrarily fixed small ε > 0. Denote by F_e and $f_e=F_e^{\prime }$ the distribution and density functions of e₀ in (2.1), respectively. The assumption (e₀) = 0 and (|e₀|) = 1 implies F_e(0) = 1/2 and F_e(1) − F_e(−1) = 1/2.

Condition 4.1

Suppose that all measurement locations x_i,j are within an interval [a, b], and order them as a = x̃₀ < x̃₁ < ··· < x̃_Nn < x̃_Nn+1 = b. Assume that

$$\underset{0\le k\le N_n}{max}\left|{\stackrel{\sim }{x}}_{k+1}-{\stackrel{\sim }{x}}_k-\frac{b-a}{N_n}\right|=O(N_n^{-2}),\text{where}{N}_n=m_1+\cdots +m_n.$$ (4.6)

Condition 4.1 requires that the pooled covariates x_i,j should be approximately uniformly dense on [a, b], which is a natural condition since otherwise it would be impossible to draw inferences for regions with very scarce observations. Pooling all subjects together is an appealing procedure to ensure this uniform denseness even though each single subject may only contain sparse measurements.

In nonparametric regression problems, there are two typical settings on the design points: fixed-design and random-design points. For fixed-design case, it is often assumed that the design points are equally spaced on some interval. For example, for the vertical density profile data of Walker and Wright (2002), the density was measured at equispaced points along a designated vertical line of wood boards. Condition 4.1 can be viewed as a generalization of the fixed-design points to allow for approximately fixed-design points. For random-design case, the design points are sampled from a distribution. For example, Assumption (a) in Appendix A of Fan and Zhang (2000)) imposed the random-design condition. In practice, both settings have different range of applicability. For example, for daily or monthly temperature series, the fixed-design setting may be appropriate; for children’s growth curve studies, it may be more reasonable to use the random-design setting since the measurements are usually taken at irregular time points.

Remark 4.2 (Asymptotic results under the random-design case)

All our subsequent theoretical results are derived under the approximate fixed-design setting in Condition 4.1, but the same argument also applies to the random-design case. Specifically, assume that the design-points {x_i,j} are random samples from a density f_X(·) with support [a, b] and that x is an interior point. Then, for the design-adaptive local linear median quantile regression estimates, the subsequent Theorems 4.1–4.2 and Corollaries 4.1–4.2 still hold with (b − a) therein replaced by 1/f_X(x). In fact, given the iid structure of {x_i,j}, the technical argument becomes much easier. For example, to establish Lemma 7.1 (again, with (b − a) therein replaced by 1/f_X(x)), elementary calculations can easily find the mean and variance for the right hand side of (7.11). All other proofs can be similarly modified and we omit the details.

Conditions 4.2–4.3 below are standard assumptions in nonparametric estimation.

Condition 4.2

K is symmetric and has bounded support and bounded derivative. Write

$${\varphi }_K={\int }_{\mathbb{R}}{K}^2(u)du\text{and}{\psi }_K=\frac{1}{2}{\int }_{\mathbb{R}}{u}^{2}K(u)du.$$

Condition 4.3

μ, s ∈ ([a, b]), inf_x∈[a,b] s(x) > 0, f_e ∈ (ℝ), f_e(0) > 0, f_e(1) +f_e(−1) > 0.

4.1. Uniform Bahadur representation for μ̂(x)

Theorem 4.1 below provides an asymptotic uniform Bahadur representation for μ̂(x) in (4.1), and its proof in Section 7.4 relies on the arguments and results in Section 3.

Theorem 4.1

Let μ̂(x) be as in (4.1). Assume that Conditions 2.1 and 4.1–4.3 hold. Further assume b_n → 0 and (log N_n)³/(N_nb_n) → 0. Then

1. We have the uniform consistency:

   $$\underset{x\in {\mathcal{S}}_{\epsilon }([a,b])}{sup}\mid \widehat{\mu }(x)-\mu (x)\mid =O_{\text{p}}\left\{b_n^2+\frac{{(log{N}_n)}^{3/2}}{{(N_n{b}_n)}^{1/2}}\right\}.$$ (4.7)
2. Moreover, the Bahadur representation

   $$\widehat{\mu }(x)-\mu (x)={\psi }_K{\rho }_{\mu }(x)b_n^2+\frac{(b-a)s(x)}{f_e(0)}\frac{Q_{b_n}(x)}{N_n{b}_n}+O_{\text{p}}(r_n)$$ (4.8)
   holds uniformly over x ∈ ([a, b]), where

   $$\begin{array}{l}{\rho }_{\mu }(x)={\mu }^{″}(x)-\left[\frac{{\mu }^{\prime }(x)f_e^{\prime }(0)}{f_e(0)}+2s^{\prime }(x)\right]\frac{{\mu }^{\prime }(x)}{s(x)},\\ Q_{b_n}(x)=-\sum _{i=1}^n\sum _{j=1}^{m_i}\{{\mathbf{1}}_{Y_{i,j}\le \mu (x)}-\mathbb{E}[{\mathbf{1}}_{Y_{i,j}\le \mu (x)}]\}{K}_{b_n}(x_{i,j}-x),\\ r_n=b_n^4+\frac{b_n^{1/2}{(log{N}_n)}^{3/2}}{N_n^{1/2}}+\frac{{(log{N}_n)}^{9/4}}{{(N_n{b}_n)}^{3/4}}.\end{array}$$

In the Bahadur representation (4.8), ${\psi }_K{\rho }_{\mu }(x)b_n^2$ is the bias term, Q_bn (x) determines the asymptotic distribution of μ̂(x) − μ(x), and r_n is the negligible error term. Such a Bahadur representation provides a powerful tool in studying the asymptotic behavior of μ̂(x). Based on Theorem 4.1, we obtain a Central Limit Theorem (CLT) for μ̂ in Corollary 4.1. Clearly, the variance of Q_bn (x) is a linear combination of K_bn (x_i,j1 − x)K_bn (x_i,j2 − x). The following regularity condition is needed to ensure the negligibility of the cross-term K_bn (x_i,j1 − x)K_bn (x_i,j2 − x) for j₁ ≠ j₂.

Condition 4.4

Assume that, for all given x ∈ ([a, b]) and k_n = O(log N_n), there exits ι_n such that k_nι_n → 0 and

$$\sum _{(i,j_1,j_2)\in \mathcal{I}}{k}_{b_n}(x_{i,j_1}-x)K_{b_n}(x_{i,j_2}-x)=O[min(h,M_n){nb}_n{k}_n{\iota }_n],M_n=\underset{1\le i\le n}{max}{m}_i,$$ (4.9)

for all h ≥ (k_n ∨ a), where = {(i, j₁, j₂) : 1 ≤ i ≤ n, a ≤ j₁ < j₂ ≤ min(a + h − 1, m_i), |j₁ − j₂| ≤ k_n}. Further assume that ${max}_j{\sum }_{i=1}^n{K}_{b_n}^r(x_{i,j}-x)=O({nb}_n)$, r = 2, 4.

Condition 4.4 is very mild. Intuitively, we consider x_i,j, j ∈ , being random locations, then $\mathbb{E}[K_{b_n}(x_{i,j_1}-x)K_{b_n}(x_{i,j_2}-x)]=O(b_n^2)$ for j₁ ≠ j₂. Thus, under the mild condition b_n log N_n → 0, (4.9) holds with ι_n = b_n.

Corollary 4.1

Let the conditions in Theorem 4.1 be fulfilled and Condition 4.4 hold. Further assume that ${(log{N}_n)}^9/(N_n{b}_n)+N_n{b}_n^9\to 0$ and nM_n = O(N_n), nb_n → ∞, $log{N}_n=O(\sqrt{M_n})$, where M_n is defined as in (4.9). Then, for any x ∈ ([a, b]), we have

$${(N_n{b}_n)}^{1/2}[\widehat{\mu }(x)-\mu (x)-{\psi }_K{\rho }_{\mu }(x)b_n^2]\Rightarrow N\left(0,\frac{{\varphi }_K(b-a)s^2(x)}{4f_e^2(0)}\right).$$ (4.10)

The proof of Corollary 4.1, given in Section 7.5, uses the coupling argument in Section 3. The condition nM_n = O(N_n) is in line with the classical CLT Lindeberg condition that none of the subjects dominates the others. If b_n is of the order $N_n^{-\beta }$, then the bandwidth condition in Corollary 4.1 holds if β ∈ (1/9, 1). By Corollary 4.1, the optimal bandwidth minimizing the asymptotic mean squared error is

$$b_n={\left[\frac{{\varphi }_K(b-a)s^2(x)}{4{\psi }_K^2{\rho }_{\mu }^2(x)f_e^2(0)}\right]}^{1/5}{N}_n^{-1/5}.$$ (4.11)

For this optimal bandwidth, the bias term is of order $O(N_n^{-2/5})$ and contains the derivatives s′, μ′, μ″ and $f_e^{\prime }$ that can be difficult to estimate. Based on the Bahadur representation (4.8), we can correct the bias term ${\psi }_K{\rho }_{\mu }(x)b_n^2$ via the jackknife estimator μ̃(x) in (4.2). Then the bias term for μ̃(x) becomes $2{\psi }_K{\rho }_{\mu }(x)b_n^2-{\psi }_K{\rho }_{\mu }(x){(\sqrt{2}{b}_n)}^2=0$. By (4.8), following the proof of Corollary 4.1, we have

$${(N_n{b}_n)}^{1/2}[\stackrel{\sim }{\mu }(x)-\mu (x)]\Rightarrow N\left(0,\frac{{\varphi }_{K^{\ast }}(b-a)s^2(x)}{4f_e^2(0)}\right),$$ (4.12)

where $K^{\ast }(u)=2K(u)-2^{-1/2}K(u/\sqrt{2})$.

4.2. Uniform Bahadur representation for ŝ(x)

Theorem 4.2 below provides a uniform Bahadur representation for ŝ(x) in (4.3).

Theorem 4.2

Let ŝ(x) be as in (4.3). Assume that the conditions in Theorem 4.1 hold. Further assume h_n + (log N_n)³/(N_nh_n) → 0. Then

1. We have the uniform consistency:

   $$\underset{x\in {\mathcal{S}}_{\epsilon }([a,b])}{sup}\mid \widehat{s}(x)-s(x)\mid =O_{\text{p}}\left\{b_n^2+h_n^2+\frac{{(log{N}_n)}^{3/2}}{{(N_n{b}_n)}^{1/2}}+\frac{{(log{N}_n)}^{3/2}}{{(N_n{h}_n)}^{1/2}}\right\}.$$ (4.13)
2. Moreover, the Bahadur representation

   $$\widehat{s}(x)-s(x)={\psi }_K{\rho }_s(x)h_n^2+(b-a)s(x)\left[\frac{W_{h_n}(x)}{N_n{h}_n{\kappa }_{+}}-\frac{\kappa T_{b_n}(x)}{N_n{b}_n{f}_e(0)}\right]+O_{\text{p}}({\stackrel{\sim }{r}}_n),$$ (4.14)
   holds uniformly over x ∈ ([a, b]), where κ₊ = f_e(−1) + f_e(1), κ = [f_e(1) − f_e(−1)]/κ₊, Q_bn (x) is defined as in Theorem 4.1,

   $$\begin{array}{l}{T}_{b_n}(x)=2Q_{b_n}(x)-2^{-1/2}{Q}_{\sqrt{2}{b}_n}(x),\\ {\rho }_s(x)=s^{″}(x)-\frac{2s^{\prime }{(x)}^2}{s(x)}+\kappa \left[{\mu }^{″}(x)-\frac{2{\mu }^{\prime }(x)s^{\prime }(x)}{s(x)}\right]-\frac{f_e^{\prime }(1){[s^{\prime }(x)+{\mu }^{\prime }(x)]}^2-f_e^{\prime }(-1){[s^{\prime }(x)-{\mu }^{\prime }(x)]}^2}{{\kappa }_{+}s(x)},\\ W_{h_n}(x)=-\sum _{i=1}^n\sum _{j=1}^{m_i}\{{\mathbf{1}}_{\mid Y_{i,j}-\mu (x)\mid \le s(x)}-\mathbb{E}[{\mathbf{1}}_{\mid Y_{i,j}-\mu (x)\mid \le s(x)}]\}{K}_{h_n}(x_{i,j}-x),\end{array}$$

   $${\stackrel{\sim }{r}}_n=b_n^4+h_n^4+\frac{h_n^{1/2}{(log{N}_n)}^{3/2}}{N_n^{1/2}}+\frac{{(log{N}_n)}^{9/4}}{{(N_n{h}_n)}^{3/4}}+\frac{{(log{N}_n)}^{9/4}}{N_n^{3/4}{b}_n^{1/4}{h}_n^{1/2}}+\frac{b_n{(log{N}_n)}^{3/2}}{{(N_n{h}_n)}^{1/2}}.$$

As in Corollary 4.1, we can use the Bahadur representation (4.14) to obtain a CLT for ŝ(x) − s(x). However, the convergence rate depends on the ratio h_n/b_n. If h_n/b_n → ∞, then the term T_bn (x)/(N_nb_n) dominates and we have (N_nb_n)^1/2-convergence; if h_n/b_n → 0, then the term W_hn (x)/(N_nh_n) dominates and we have (N_nh_n)^1/2-convergence; if h_n/b_n → c for a constant c ∈ (0, ∞), then both terms contribute.

Corollary 4.2

Let the conditions in Theorem 4.2 be fulfilled and Condition 4.4 and its counterpart version with b_n being replaced by h_n hold. Further assume that

$$N_n{(b_n\vee h_n)}^9+\frac{{(log{N}_n)}^9}{N_n(b_n\wedge h_n)}\to 0,$$

and nM_n = O(N_n), n(b_n ∧ h_n) → ∞, $log{N}_n=O(\sqrt{M_n})$, where M_n is defined as in (4.9). Recall $K^{\ast }(u)=2K(u)-2^{-1/2}K(u/\sqrt{2})$ in (4.12) and κ, κ₊ in Theorem 4.2. Let x ∈ ([a, b]) be a fixed point. Suppose h_n/b_n → c.

1. If κ ≠ 0 and c = ∞, then

   $${(N_n{b}_n)}^{1/2}[\widehat{s}(x)-s(x)-{\psi }_K{\rho }_s(x)h_n^2]\Rightarrow N\left(0,\frac{{\varphi }_{K^{\ast }}{\kappa }^2(b-a)s^2(x)}{4f_e^2(0)}\right).$$
2. If κ ≠ 0 and c ∈ [0, ∞), then

   $${(N_n{h}_n)}^{1/2}[\widehat{s}(x)-s(x)-{\psi }_K{\rho }_s(x)h_n^2]\Rightarrow N(0,{\sigma }_c^2),$$ (4.15)
   where

   $${\sigma }_c^2=\frac{(b-a)s^2(x)}{4}\left\{\frac{{\varphi }_K}{{\kappa }_{+}^2}+\frac{c^2{\kappa }^2{\varphi }_{K^{\ast }}}{f_e^2(0)}-\frac{2c\kappa [1-4F_e(-1)]}{{\kappa }_{+}{f}_e(0)}{\int }_{\mathbb{R}}K(u)K^{\ast }(cu)du\right\}.$$

3. If κ = 0, then for all c ∈ [0, ∞], (4.15) holds with ${\sigma }_c^2={\varphi }_K(b-a)s^2(x)/(4{\kappa }_{+}^2)$.

One can similarly establish CLT results for s̃(x) in (4.4). We omit the details.

5. An illustration using real data

5.1. Bandwidth selection

For least-squares based estimation of longitudinal data, Rice and Silverman (1991) suggested the subject-based cross-validation method. The basic idea is to use all but one subject to do model fitting, validate the fitted model using the left-out subject, and finally choose the optimal bandwidth by minimizing the overall prediction error:

$$b_{\text{LS}}^{\ast }=\underset{b}{\text{argmin}}\sum _{i=1}^n\sum _{j=1}^{m_i}{\{Y_{i,j}-{\stackrel{\sim }{\mu }}^{(-i)}(x_{i,j})\}}^2,$$ (5.1)

where μ̃⁽⁻ⁱ⁾(x) represents the estimator of μ(x) based on data from all but ith subject. As in Wei, Zhao and Lin (2012), we replace the square loss by absolute deviation:

$$b_{\text{LAD}}^{\ast }=\underset{b}{\text{argmin}}\sum _{i=1}^n\sum _{j=1}^{m_i}\mid Y_{i,j}-{\stackrel{\sim }{\mu }}^{(-i)}(x_{i,j})\mid .$$ (5.2)

5.2. An illustration using progesterone data

Urinary metabolite progesterone levels are measured daily, around the ovulation day, over 22 conceptive and 69 nonconceptive women’s menstrual cycles so that each curve has about 24 design points; see the left panel of Figure 1 for a plot of the trajectories of the 22 conceptive women. Previous studies based on least-squares (LS) methods include Brumback and Rice (1998), Fan and Zhang (2000), and Wu and Zhang (2002). Here we re-analyze the conceptive group using our least-absolute-deviation (LAD) estimates.

Figure 1.

Left: Trajectories of the measurements from 22 conceptive women. Right: Estimates of μ(·) using both the original data and perturbed data. Thin solid, dotted, and dashed curves are the least-squares estimates of μ(·) based on the original data, perturbation scenario I (remove subjects 13 and 14), and perturbation scenario II (shift subjects 13 and 14 down by three units), respectively. Similarly, thick solid, dotted, and dashed curves are least-absolute-deviation estimates.

From the left plot in Figure 1, subject 14 (dashed curve) has two sharp drops in progesterone levels at days −3 and 9. Similarly, subject 13 (dotted curve) has unusually low levels on days −1, 0, 1. While such sharp drops or “outliers” may be caused by incorrect measurements or other unknown reasons, we investigate the impact of such “outliers” on the LS and LAD estimates. In the right plot of Figure 1, the thick solid and thin solid curves are the LAD and LS estimates of μ(·). The two estimates are reasonably close except during the periods [−4, 1] and [8, 15]. Notice that the latter periods contain the “outliers” from subjects 13, 14.

To understand the impact of such possible “outliers”, we consider two scenarios of perturbing the data below.

1. Scenario I: remove subjects 13 and 14 and estimate μ(·) using the remaining subjects. The thick dotted and thin dotted curves are the corresponding LAD and LS estimates. Clearly, the discrepancy is largely diminished.

2. Scenario II: make the two outlier subjects 13 and 14 even more extreme by shifting their curves three units down. We see that the discrepancy between the LAD (thick dashed) and LS (thin dashed) estimates becomes even more remarkable.

Compared with the estimate based on the original data, the LS estimates under the two perturbation scenarios differ significantly. By contrast, the LAD estimates under the three cases are similar, indicating the robustness in the presence of outliers. We conclude that, for the progesterone data with several possible outliers, the proposed LAD estimate offers an attractive alternative over the well-studied LS estimates. In practice, we recommend the LAD estimate if the data has suspicious, unusual observations or extreme outliers.

6. Conclusion and extension to spatial setting

This paper studies robust estimations of the location and scale functions in a nonparametric regression model with serially dependent data from multiple subjects. Under a general error dependence structure that allows for many linear and nonlinear processes, we study uniform Bahadur representations and asymptotic normality for least-absolute-deviation estimations of a location-scale longitudinal model. In the large literature on nonparametric estimation of longitudinal models, most existing works use least-squares based methods, which are sensitive to extreme observations and may perform poorly in such circumstances. Despite the popularity of quantile regression methods in linear models and nonparametric regression models, little research has been done in quantile regression based estimations for nonparametric longitudinal models, partly due to difficulties in dealing with the dependence. Therefore, our work provides a solid theoretical foundation for quantile regression estimations in longitudinal models.

The study of asymptotic Bahadur representations is a difficult area and has focused mainly on the iid setting or stationary time series setting. For longitudinal data, deriving Bahadur representations is more challenging due to the non-stationarity and dependence. To obtain our Bahadur representations, we develop substantial theory for kernel weighted empirical processes via a coupling argument.

The proposed error dependence structure and coupling argument provide a flexible and powerful framework for asymptotics from dependent data, such as time series data, longitudinal data and spatial data, whereas similar problems have been previously studied mainly for either independent data or stationary time series. In (2.1), e_j depends on the innovations or shocks ε_j, ε_j±1, …, indexed by integers on a line. A natural extension is the function of innovations indexed by bivariate integers on a square:

$$e_j=G({\epsilon }_{j,j},{\epsilon }_{j,j\pm 1},{\epsilon }_{j\pm 1,j},{\epsilon }_{j\pm 1,j\pm 1},\dots ),j\in \mathbb{Z}.$$

The coupling argument still holds by replacing the innovations ε_j±r,j±s, r, s ≥ k + 1, outside the k nearest squares with iid copies. As in Condition 2.1, we can assume that the impact of perturbing the distant innovations decays exponentially fast (or polynomially fast with slight modifications of the proof). More generally, the coupling argument holds for function of innovations indexed by multivariate spatial lattice, and such setting may be useful in studying asymptotics for spatial data.

7. Technical proofs

Throughout c, c₁, c₂, …, are generic constants. First, we give an inequality for the indicator function. Let Z, Z′ be two random variables and y ∈ ℝ. For α > 0, we have

$${\mathbf{1}}_{Z\le y<Z^{\prime }}={\mathbf{1}}_{Z\le y<Z^{\prime },\mid Z-Z^{\prime }\mid \ge \alpha }+{\mathbf{1}}_{Z\le y<Z^{\prime },\mid Z-Z^{\prime }\mid <\alpha }\le {\mathbf{1}}_{\mid Z-Z^{\prime }\mid \ge \alpha }+{\mathbf{1}}_{y<Z^{\prime }<y+\alpha },$$

Similarly, 1_Z′≤y<z ≤ 1_{|Z−Z′|≥α} + 1_{y−α<Z′≤y}. Therefore,

$$\mid {\mathbf{1}}_{Z\le y}-{\mathbf{1}}_{Z^{\prime }\le y}\mid ={\mathbf{1}}_{Z\le y<Z^{\prime }}+{\mathbf{1}}_{Z^{\prime }\le y<Z}\le 2{\mathbf{1}}_{\mid Z-Z^{\prime }\mid \ge \alpha }+{\mathbf{1}}_{y-\alpha <Z^{\prime }<y+\alpha }.$$ (7.1)

7.1. Proof of Propositions 2.1–2.2

Proof of Proposition 2.1

Let q^* = q/(ς + υ), p₁ = υ/ς + 1, and p₂ = ς/υ + 1 so that ςq^*p₁ = q, υq^*p₂ = q, and 1/p₁ + 1/p₂ = 1. For convenience write $e_0^{\prime }=e_0(k)$. By assumption, ${\Vert e_0-e_0^{\prime }\Vert }_q=O({\rho }^k)$. By (2.3) and the Hölder inequality |Z₁Z₂| ≤ ||Z₁||_p1 ||Z₂||_p2,

$$\begin{array}{l}{\Vert h(e_0^{\prime })-h(e_0)\Vert }_{q^{\ast }}^{q^{\ast }}\le O(1)\mathbb{E}[{\mid e_0^{\prime }-e_0\mid }^{\varsigma q^{\ast }}{(1+\mid e_0\mid +\mid e_0^{\prime }\mid )}^{\upsilon q^{\ast }}]\\ \le O(1){\{\mathbb{E}[{\mid e_0-e_0^{\prime }\mid }^{\varsigma q^{\ast }·p_1}]\}}^{1/p_1}{\{\mathbb{E}[{(1+\mid e_0\mid +\mid e_0^{\prime }\mid )}^{\upsilon q^{\ast }·p_2}]\}}^{1/p_2}\\ =O(1){\Vert e_0-e_0^{\prime }\Vert }_q^{q/p_1}{\Vert e_0\Vert }_q^{q/p_2}=O({\rho }^{kq/p_1}).\end{array}$$

The above expression gives ${\Vert h(e_0^{\prime })-h(e_0)\Vert }_{q^{\ast }}\le O(1){[{\rho }^{q/(p_1{q}^{\ast })}]}^k=O({\rho }^{k\varsigma })$.

Proof of Proposition 2.2

Let α = ρ^kq/(1+q). By (7.1) and the triangle inequality,

$$\begin{array}{l}{\Vert {\mathbf{1}}_{e_0\le x}-{\mathbf{1}}_{e_0(k)\le x}\Vert }_q\le 2{\Vert {\mathbf{1}}_{\mid e_0-e_0(k)\mid \ge \alpha }\Vert }_q+{\Vert {\mathbf{1}}_{x-\alpha \le e_0\le x+\alpha }\Vert }_q\\ =2{[\mathbb{P}\{\mid e_0-e_0(k)\mid \ge \alpha \}]}^{1/q}+{[\mathbb{P}\{x-\alpha \le e_0\le x+\alpha \}]}^{1/q}.\end{array}$$

By the Markov inequality, ℙ{|e₀ − e₀(k)| ≥ α} = [|e₀ − e₀(k)|^q]/α^q = O(ρ^kq/α^q). Since e₀ has a bounded density, ℙ{x − α ≤ e₀ ≤ x + α} = O(α). The result then follows.

7.2. Proof of Theorems 3.1–3.3

Proof of Theorem 3.1

for r = 1, 2, …, 2k_n + 1, let

$${\mathcal{I}}_r=\{(i,j):1\le i\le n,1\le j\le \lfloor (m_i-r)/(2k_n+1)\rfloor +1\}.$$ (7.2)

Using the identity ${\sum }_{j=1}^m{a}_j={\sum }_{r=1}^k{\sum }_{j=1}^{\lfloor (m-r)/k\rfloor +1}{a}_{(j-1)k+r}$ for all k, m ∈ ℕ, a₁, …, a_m ∈ ℝ, we can rewrite H_n(v) as

$$H_n(v)=\sum _{r=1}^{2k_n+1}\sum _{(i,j)\in {\mathcal{I}}_r}{g}_{i,(j-1)(2k_n+1)+r}({\stackrel{\sim }{Y}}_{i,(j-1)(2k_n+1)+r},v):=\sum _{r=1}^{2k_n+1}{H}_n(v,r).$$ (7.3)

Now we consider H_n(v, r). By the discussion in Section 3, the summands in H_n(v, r) are independent. By (3.3),

$$\begin{array}{l}\text{Var}[H_n(v,r)]=\sum _{(i,j)\in {\mathcal{I}}_r}\mathbb{E}[g_{i,(j-1)(2k_n+1)+r}^2({\stackrel{\sim }{Y}}_{i,(j-1)(2k_n+1)+r},v)]\\ \le \sum _{i=1}^n\sum _{j=1}^{m_i}\mathbb{E}[g_{i,j}^2({\stackrel{\sim }{Y}}_{i,j},v)]\le {\chi }_n,\end{array}$$ (7.4)

uniformly over v, r.

1. Consider the case χ_n = O(1). Recall the condition |g_i,j(y, v)| ≤ c. By Berstein’s exponential inequality (Bennett, 1962) for bounded and independent random variables, for any given c₁ > 0, when N_n is sufficiently large,

   $$\mathbb{P}\{\mid H_n(v,r)\mid \ge c_{1}log{N}_n\}\le 2exp\left\{-\frac{{(c_{1}log{N}_n)}^2}{2\text{Var}[{\mathrm{\Lambda }}_n(r,h)]+{cc}_{1}log{N}_n}\right\}\le 2N_n^{-c_1/(3c)},$$ (7.5)
   uniformly over r and h. Here the second inequality follows from Var[H_n(v, r)] ≤ χ_n = O(1) ≤ cc₁ log N_n for large enough N_n. Thus,

   $$\begin{array}{l}\mathbb{P}\left\{\underset{v\in {\mathcal{V}}_n,1\le r\le 2k_n+1}{max}\mid H_n(v,r)\mid \ge c_{1}log{N}_n\right\}\le \sum _{v\in {\mathcal{V}}_n,1\le r\le 2k_n+1}\mathbb{P}\{\mid H_n(v,r)\mid \ge c_{1}log{N}_n\}\\ \le 2\mid {\mathcal{V}}_n\mid K_n{N}_n^{-c_1/(3c)}.\end{array}$$
   By the assumption that both | | and k_n grow no faster than a polynomial of N_n, we can make the above probability go to zero by choosing a large enough c₁. Therefore, max_{v∈ ,1≤r≤2kn+1} |H_n(v, r)| = O_p(log N_n). By (7.3), the desired result follows from

   $$\underset{v\in {\mathcal{V}}_n}{max}\mid H_n(v)\mid \le (2k_n+1)\underset{v\in {\mathcal{V}}_n,1\le r\le 2k_n+1}{max}\mid H_n(v,r)\mid .$$
2. Consider the case sup_n log N_n/χ_n < ∞. As in (7.5),

   $$\mathbb{P}\{\mid H_n(v,r)\mid \ge c_1\sqrt{{\chi }_{n}log{N}_n}\}\le 2exp\{-\frac{{(c_1\sqrt{{\chi }_{n}log{N}_n})}^2}{2{\chi }_n+{cc}_1\sqrt{{\chi }_{n}log{N}_n}}\}=O[N_n^{-c_1^2/(2+{cc}_1{c}_2)}],$$

   uniformly over r and h, where c₂ = sup_n[log N_n/χ_n]^1/2 < ∞. The rest of the proof follows from the same argument as in the case (i) by choosing a sufficiently large c₁.

Proof of Theorem 3.2

Let α = 1/N_n. Since ϖ_i,j(x) ≤ c, applying (7.1), we obtain

$$\begin{array}{l}\mid F_n(x,y)-{\stackrel{\sim }{F}}_n(x,y)\mid \le \sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}(x)\mid {\mathbf{1}}_{Y_{i,j}\le y}-{\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le y}\mid \\ \le 2c\left[\sum _{i=1}^n\sum _{j=1}^{m_i}{\mathbf{1}}_{\mid Y_{i,j}-{\stackrel{\sim }{Y}}_{i,j}\mid \ge \alpha }+\sum _{i=1}^n\sum _{j=1}^{m_i}{\mathbf{1}}_{y-\alpha <{\stackrel{\sim }{Y}}_{i,j}<y+\alpha }\right]\\ :=2c[{\mathrm{\Omega }}_n+{\mathrm{\Lambda }}_n(y)].\end{array}$$ (7.6)

Notice that, |Y_i,j − Ỹ_i,j| = O(1)|e_i,j − e_i,j(k_n)|. By (2.2) and the Markov inequality,

$$\mathbb{E}({\mathbf{1}}_{\mid Y_{i,j}-{\stackrel{\sim }{Y}}_{i,j}\mid \ge \alpha })\le \frac{{\Vert Y_{i,j}-{\stackrel{\sim }{Y}}_{i,j}\Vert }_q^q}{{\alpha }^q}=O(1)\frac{{\Vert e_{i,j}-e_{i,j}(k_n)\Vert }_q^q}{{\alpha }^q}=O(N_n^q{\rho }^{{qk}_n}).$$

Thus, ${\mathrm{\Omega }}_n=O_{\text{p}}(N_n^{1+q}{\rho }^{{qk}_n})=O_{\text{p}}[N_n^{1+q}{\rho }^{q\lambda log(N_n)}]=o_{\text{p}}(1)$ for λ > (q + 1)/[q log(1/ρ)].

For Λ_n(y) over y ∈ ℝ, consider two cases: $\mid y\mid >N_n^{1/q}$ and $\mid y\mid \le N_n^{1/q}$. For $\mid y\mid >N_n^{1/q}$, since α = 1/N_n → 0, μ(x_i,j) and s(x_i,j) are bounded, $\{y-\alpha <{\stackrel{\sim }{Y}}_{i,j}<y+\alpha \}\subset \{\mid e_{i,j}(k_n)\mid \ge c_1{N}_n^{1/q}\}$ for some constant c₁ > 0. Therefore, by e_i,j(k_n) ∈ and the Markov inequality,

$$\mathbb{E}\left[\underset{\mid y\mid >N_n^{1/q}}{sup}{\mathrm{\Lambda }}_n(y)\right]\le \mathbb{E}\left[\sum _{i=1}^n\sum _{j=1}^{m_i}{\mathbf{1}}_{\mid e_{i,j}(k_n)\mid >c_1{N}_n^{1/q}}\right]\le \sum _{i=1}^n\sum _{j=1}^{m_i}\frac{{\Vert e_{i,j}(k_n)\Vert }_q^q}{{(c_1{N}_n^{1/q})}^q}=O(1).$$ (7.7)

We conclude that ${sup}_{\mid y\mid >N_n^{1/q}}{\mathrm{\Lambda }}_n(y)=O_{\text{p}}(1)$.

In what follows we use a chain argument to prove ${sup}_{y\in [-N_n^{1/q},N_n^{1/q}]}{\mathrm{\Lambda }}_n(y)=O_{\text{p}}[{(logn)}^2]$. Without loss of generality, consider $y\in [0,N_n^{1/q}]$. Write ${\ell }_n=[N_n^{1+1/q}]$ and let ${\mathcal{V}}_n=\{y_v={vN}_n^{1/q}/{\ell }_n,v=0,1,\dots ,{\ell }_n\}$ be the set of ℓ_n + 1 grid points uniformly spaced over $[0,N_n^{1/q}]$. Partition $[0,N_n^{1/q}]$ into intervals I_v = [y_v−1, y_v], v = 1, …, ℓ_n. For any y ∈ I_v, we have 1_{y−α<Ỹi,jy+α} ≤ 1_{yv−1−α<Ỹi,jyv+α}. Since s(x_i,j) is bounded away from zero, sup_u f_e(u) < ∞, and |y_v − y_v−1| = O(1/N_n), we have (1_{yv−1−α<Ỹi,jyv+α}) ≤ c₂/N_n uniformly for some constant c₂ < ∞. Consequently, for any y ∈ I_v, we have

$${\mathrm{\Lambda }}_n(y)\le \sum _{i=1}^n\sum _{j=1}^{m_i}\left[\{{\mathbf{1}}_{y_{v-1}-\alpha <{\stackrel{\sim }{Y}}_{i,j}<y_v+\alpha }-\mathbb{E}({\mathbf{1}}_{y_{v-1}-\alpha <{\stackrel{\sim }{Y}}_{i,j}<y_v+\alpha })\}+c_2/N_n\right]={\mathrm{\Lambda }}_n^{\ast }(v)+c_2.$$

We apply Theorem 3.1 to ${\mathrm{\Lambda }}_n^{\ast }(v)$. For χ_n in (3.3), using (1_{yv−1−α<Ỹi,jyv+α}) ≤ c₂/N_n, we have χ_n = O(1) and thus ${max}_{v\in {\mathcal{V}}_n}\mid {\mathrm{\Lambda }}_n^{\ast }(v)\mid =O_{\text{p}}[{(log{N}_n)}^2]$, completing the proof.

Proof of Theorem 3.3

Recall the coupling process D̃_n(δ, x, y) in (3.7). Under the assumption sup_n log N_n/(δ_nφ_n) < ∞, (log N_n)² = O{[δ_nφ_n(log N_n)³]^1/2}. Thus, by (3.8), it suffices to show sup_{|δ|≤δn,x,y∈ℝ}|D̃_n(δ, x, y)| = O_p{[δ_nφ_n(log N_n)³]^1/2}.

Without loss of generality assume δ ∈ [0, δ_n]. Recall Ỹ_i,j in (3.5). Rewrite

$${\stackrel{\sim }{D}}_n(\delta ,x,y)=\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}(x)\{{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)-\mathbb{E}[{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)]\},{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)={\mathbf{1}}_{y<{\stackrel{\sim }{Y}}_{i,j}\le y+\delta }.$$

As in the proof of Theorem 3.2, consider $\mid y\mid >N_n^{1/q}$ and $\mid y\mid \le N_n^{1/q}$.

For $\mid y\mid >N_n^{1/q}$, since μ(x_i,j) and s(x_i,j) are bounded and |δ| ≤ δ_n → 0, $\{y<{\stackrel{\sim }{Y}}_{i,j}\le y+\delta \}\subset \{\mid e_{i,j}(k_n)\mid \ge c_1{N}_n^{1/q}\}$ for some c₁ > 0. Therefore, by the boundedness of ϖ_i,j (·), the same argument in (7.7) shows D̃_n(δ, x, y) = O_p(1) uniformly over x ∈ ℝ, $\mid y\mid >N_n^{1/q}$, |δ| ≤ δ_n.

Next we consider $\mid y\mid \le N_n^{1/q}$. Since ϖ_i,j(x) vanishes for x outside a bounded interval, without loss of generality we only consider x ∈ [0, b] for some b > 0, $y\in [0,N_n^{1/q}]$, and δ ∈ [0, δ_n]. As in the proof of Theorem 3.2, we use the chain argument. Let ${\ell }_n=\lfloor N_n^{1/q}/{\delta }_n+N_n{\tau }_n+N_n^{1+1/q}\rfloor$, and

$${\mathcal{V}}_n=\left\{(x_{v_1},y_{v_2},t_{v_3}),x_{v_1}=\frac{v_{1}b}{{\ell }_n},y_{v_2}=\frac{v_2{N}_n^{1/q}}{{\ell }_n},t_{v_3}=\frac{v_3{\delta }_n}{{\ell }_n},v_1,v_2,v_3=0,1,\dots .{\ell }_n\right\}$$

be uniformly spaced grid points. Partition $[0,b]\times [0,N_n^{1/q}]\times [0,{\delta }_n]$ into intervals I_v1v2,v3 = [x_v1−1, x_v1] × [y_v2−1, y_v2] × [t_v3−1, t_v3], v₁, v₂, v₃ = 1, …, ℓ_n. Let

$${\underset{\_}{\xi }}_{i,j}(v_2,v_3)={\mathbf{1}}_{y_{v_2}}<{\stackrel{\sim }{Y}}_{i,j}\le y_{v_2-1}+t_{v_3-1}\text{and}{\overline{\xi }}_{i,j}(v_2,v_3)={\mathbf{1}}_{y_{v_2-1}<{\stackrel{\sim }{Y}}_{i,j}\le y_{v_2}+t_{v_3}}.$$

Clearly, for any (x, y, δ) ∈ I_{v1,v2, v3}, we have ξ_i,j(v₂, v₃) ≤ ξ̃_i,j(δ, y) ≤ ξ̄_i,j (v₂, v₃). Since N_n → ∞ and δ_n → 0, there exists a constant c₂ < ∞ such that $0\le \mathbb{E}[{\overline{\xi }}_{i,j}(v_2,v_3)]-\mathbb{E}[{\underset{\_}{\xi }}_{i,j}(v_2,v_3)]\le c_2{N}_n^{1/q}/{\ell }_n$. Additionally, for x ∈ [x_v1−1, x_v1], by Condition 3.2, |ϖ_i,j(x) − ϖ_i,j (x_v1) | ≤ τ_n|x − x_v1 | ≤ τ_nb/ℓ_n. Thus, there exists a constant c₃ < ∞ such that

$$\begin{array}{l}{\varpi }_{i,j}(x)\{{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)-\mathbb{E}[{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)]\}\le {\varpi }_{i,j}(x_{v_1})\{{\overline{\xi }}_{i,j}(v_2,v_3)-\mathbb{E}[{\underset{\_}{\xi }}_{i,j}(v_2,v_3)]\}+{\tau }_{n}b/{\ell }_n\\ \le {\varpi }_{i,j}(x_{v_1})\{{\overline{\xi }}_{i,j}(v_2,v_3)-\mathbb{E}[{\overline{\xi }}_{i,j}(v_2,v_3)]\}+c_3({\tau }_n+N_n^{1/q})/{\ell }_n,\end{array}$$ (7.8)

uniformly over i, j, and (x, y, δ) ∈ I_v1,v2,v3. Similarly,

$${\varpi }_{i,j}(x)\{{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)-\mathbb{E}[{\stackrel{\sim }{\xi }}_{i,j}(\delta ,y)]\}\ge {\varpi }_{i,j}(x_{v_1})\{{\underset{\_}{\xi }}_{i,j}(v_2,v_3)-\mathbb{E}[{\underset{\_}{\xi }}_{i,j}(v_2,v_3)]\}-c_3({\tau }_n+N_n^{1/q})/{\ell }_n.$$ (7.9)

Combining (7.8) and (7.9) and using $N_n({\tau }_n+N_n^{1/q})/{\ell }_n=O(1)$, we have

$$\underset{x,y,\delta }{sup}\mid {\stackrel{\sim }{D}}_n(\delta ,x,y)\mid \le \underset{v\in {\mathcal{V}}_n}{max}\{\mid {\underset{\_}{\mathrm{\Delta }}}_n(v)\mid +\mid {\overline{\mathrm{\Delta }}}_n(v)\mid \}+O(1),$$ (7.10)

where v = (v₁, v₂, v₃),

$$\begin{array}{l}{\underset{\_}{\mathrm{\Delta }}}_n(v)=\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{ij}(x_{v_1})\{{\underset{\_}{\xi }}_{i,j}(v_2,v_3)-\mathbb{E}[{\xi }_{i,j}(v_2,v_3)]\},\\ {\overline{\mathrm{\Delta }}}_n(v)=\sum _{i=1}^n\sum _{j=1}^{m_i}{\varpi }_{i,j}(x_{v_1})\{{\overline{\xi }}_{i,j}(v_2,v_3)-\mathbb{E}[{\overline{\xi }}_{i,j}(v_2,v_3)]\}.\end{array}$$

We now apply Theorem 3.1 to Δ_n(v) and Δ̄_n(v). For χ_n in (3.3), with φ_n in (3.9) and $\mathbb{E}[{\overline{\xi }}_{i,j}(h_2,h_3)]=O({\delta }_n+N_n^{1/q}/{\ell }_n)=O({\delta }_n)$, we can take χ_n = O(δ_nφ_n). By Theorem 3.1 (ii), max_v∈ |Δ̄_n(v) = O_p{[δ_nφ_n(log N_n)³]^1/2}. The latter bound also holds for max_v∈ |Δ_n(v)|. The desired result then follows from (7.10).

7.3. Asymptotic expansions

Throughout the proofs, we use the following notations:

$$\begin{array}{l}{L}_{\mu }({\delta }_1,x)=\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{b_n}(x_{i,j}-x){\mathbf{1}}_{Y_{i,j}\le \mu (x)+{\delta }_1},\\ L_{\mu }(x)=\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{b_n}(x_{i,j}-x),\\ J_{\mu }({\delta }_1,x)=\mathbb{E}[L_{\mu }({\delta }_1,x)],\\ L_s({\delta }_1,{\delta }_2,x)=\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{h_n}(x_{i,j}-x){\mathbf{1}}_{\mid Y_{i,j}-\mu (x)-{\delta }_1\mid \le s(x)+{\delta }_2},\\ L_s(x)=\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{h_n}(x_{i,j}-x),\\ J_s({\delta }_1,{\delta }_2,x)=\mathbb{E}[L_s({\delta }_1,{\delta }_2,x)].\end{array}$$

Lemma 7.1

Assume that Conditions 4.1–4.2 hold. Then, we have

1. Uniformly over x ∈ [a, b],

   $$\sum _{i=1}^n\sum _{j=1}^{m_i}{\left(\frac{x_{i,j}-x}{b_n}\right)}^{r}K\left(\frac{x_{i,j}-x}{b_n}\right)=\frac{N_n{b}_n}{b-a}{\int }_{\mathbb{R}}{u}^{r}K(u)du+O(1).$$ (7.11)
2. Let g(x, v) be a measurable bivariate function on [a, b]². Define

   $${\mathcal{G}}_g(x)=\sum _{i=1}^n\sum _{j=1}^{n_i}g(x,x_{i,j})K_{b_n}(x_{i,j}-x).$$ (7.12)
   Further assume that sup_x∈[a,b] |∂^s(x, v)/∂v^s| < ∞, s = 0, 1, …, r for some r ∈ ℕ. Then uniformly over x ∈ [a, b],

   $${\mathcal{G}}_g(x)=\sum _{s=0}^{r-1}{\frac{{\partial }^{s}g(x,v)}{\partial v^s}|}_{v=x}\frac{N_n{b}_n^{s+1}}{(b-a)s!}{\int }_{\mathbb{R}}{u}^{s}K(u)du+O(1+N_n{b}_n^{r+1}).$$ (7.13)

Proof

1. Recall the ordered locations x̃_k in Condition 4.1. Define

   $$S_n(x)=\sum _{k=1}^{N_n}{\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)}^{r}K\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right),$$ (7.14)

   $$I_n(x)=\sum _{k=0}^{N_n}({\stackrel{\sim }{x}}_{k+1}-{\stackrel{\sim }{x}}_k){\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)}^{r}K\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right),$$ (7.15)

   $${\varrho }_n=\underset{0\le k\le N_n}{max}\left|{\stackrel{\sim }{x}}_{k+1}-{\stackrel{\sim }{x}}_k-(b-a)/N_n\right|=O(N_n^{-2}),$$ (7.16)

   $$\mathcal{I}(x)=\{1\le k\le N_n:{\stackrel{\sim }{x}}_k-x\in [-b_n-(b-a)/N_n-{\varrho }_n,b_n]\}.$$ (7.17)
   Assume without loss of generality that K has support [−1, 1]. Condition (4.6) implies that sup_x∈[a,b] | (x)| = O(N_nb_n), where and hereafter | | is the cardinality of a set . Because K has support [−1, 1], K_bn (x̃_k − x) = 0 for k ∉ (x). Additionally, for k ∈ (x), the summands in S_n(x) are uniformly bounded. Thus,

   $$S_n(x)=\sum _{k\in \mathcal{I}(x)}{\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)}^{r}K\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)=O[\mid \mathcal{I}(x)\mid ]=O(N_n{b}_n),$$ (7.18)

   uniformly over x ∈ [a, b].

   By (4.6), elementary calculation shows that, uniformly over x ∈ [a, b],

   $$\begin{array}{l}\frac{b-a}{N_n}{S}_n(x)-I_n(x)=-\sum _{k=1}^{N_n}\left({\stackrel{\sim }{x}}_{k+1}-{\stackrel{\sim }{x}}_k-\frac{b-a}{N_n}\right){\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)}^{r}K\left(\frac{{\stackrel{\sim }{x}}_k-x}{b_n}\right)\\ =O({\varrho }_n)\underset{x\in [a,b]}{sup}\mid S_n(x)\mid =O(b_n/N_n).\end{array}$$ (7.19)
   Write u_k = (x̃_k − x)/b_n. Observe that $I_n(x)={\sum }_{k=0}^{N_n}{\int }_{{\stackrel{\sim }{x}}_k}^{{\stackrel{\sim }{x}}_{k+1}}{u}_k^{r}K(u_k)dv$. Thus, by the triangle inequality, we have

   $$\begin{array}{l}\left|I_n(x)-{\int }_{{\stackrel{\sim }{x}}_0}^{{\stackrel{\sim }{x}}_{N_n+1}}{\left(\frac{v-x}{b_n}\right)}^{r}K\left(\frac{v-x}{b_n}\right)dv\right|\le \sum _{k=0}^{N_n}{V}_k,\\ \text{where}{V}_k={\int }_{{\stackrel{\sim }{x}}_k}^{{\stackrel{\sim }{x}}_{k+1}}\left|u_k^{r}K(u_k)-{\left(\frac{v-x}{b_n}\right)}^{r}K\left(\frac{v-x}{b_n}\right)\right|dv.\end{array}$$ (7.20)
   Since K has bounded derivative, |y^rK(y) − z^rK(z)| = O(|y − z|) for y, z ∈ [−1, 1]. Also, |u_k − (v − x)/b_n| = |v − x̃_k|/b_n. Thus, under Condition 4.1,

   $$\mid V_k\mid =O(1){\int }_{{\stackrel{\sim }{x}}_k}^{{\stackrel{\sim }{x}}_{k+1}}\frac{v-{\stackrel{\sim }{x}}_k}{b_n}dv=\frac{O[{({\stackrel{\sim }{x}}_{k+1}-{\stackrel{\sim }{x}}_k)}^2]}{b_n}\frac{O(1)}{N_n^2{b}_n}.$$ (7.21)
   Furthermore, it is easily seen that, for k ∉ (x), min(|x̃_k − x|, |x̃_k+1 − x|) > b_n, which implies K(u_k) = 0, K{(v − x)/b_n} = 0 for v ∈ [x̃_k, x̃_k+1], and consequently V_k = 0. Thus, by (7.20) and (7.21),

   $$\left|I_n(x)-{\int }_{{\stackrel{\sim }{x}}_0}^{{\stackrel{\sim }{x}}_{N_n+1}}{\left(\frac{v-x}{b_n}\right)}^{r}K\left(\frac{v-x}{b_n}\right)dv\right|\le \sum _{k\in \mathcal{I}(x)}{V}_k=O(1/N_n),$$ (7.22)

   uniformly over x ∈ [a, b],

   Notice that ${\sum }_{i=1}^n{\sum }_{j=1}^{m_i}{[(x_{i,j}-x)/b_n]}^r{K}_{b_n}(x_{i,j}-x)=S_n(x)$. Recall that x̃₀ = a and x̃_Nn+1 = b. The desired result then follows from (7.19) and (7.22) in view of

   $${\int }_{{\stackrel{\sim }{x}}_0}^{{\stackrel{\sim }{x}}_{N_n+1}}{\left(\frac{v-x}{b_n}\right)}^{r}K\left(\frac{v-x}{b_n}\right)dv=b_n{\int }_{(a-x)/b_n}^{(b-x)/b_n}{u}^{r}K(u)du=b_n{\int }_{-1}^1{u}^{r}K(u)du,$$

   for all x ∈ [a, b] and large enough n.

2. The expression (7.13) easily follows from (i) in view of the Taylor expansion $g(x,x_{i,j})={\sum }_{s=0}^{r-1}{\partial }^{s}g(x,v)/\partial v^s{\mid }_{v=x}{(x_{i,j}-x)}^s/s!+O(b_n^r)$ for |x_i,j − x| ≤ b_n.

Lemma 7.2

Assume that Conditions 4.1–4.2 hold. Let ρ_μ(x), ρ_s(x), κ, κ₊ be as in Theorems 4.1–4.2. Then, for δ₁ → 0, δ₂ → 0, we have uniformly over x ∈ [a, b],

$$\begin{array}{l}{J}_{\mu }(0,x)=L_{\mu }(x)/2-N_n{b}_n^3{\rho }_{\mu }(x)f_e(0){\psi }_K/[(b-a)s(x)]+O(1+N_n{b}_n^5),\\ J_{\mu }({\delta }_1,x)=J_{\mu }(0,x)+N_n{b}_n{\delta }_1\{f_e(0)/[(b-a)s(x)]+O[{(N_n{b}_n)}^{-1}+b_n^2+{\delta }_1]\},\\ J_s({\delta }_1,0,x)=L_s(x)/2-N_n{h}_n{\kappa }_{+}\{[h_n^2{\psi }_K{\rho }_s(x)-{\delta }_1\kappa ]/[(b-a)s(x)]+O(h_n^4+{\delta }_1^2)\},\\ J_s({\delta }_1,{\delta }_2,x)=J_s({\delta }_1,0,x)+N_n{h}_n{\delta }_2\{{\kappa }_{+}/[(b-a)s(x)]+O(h_n^2+{\delta }_1+{\delta }_2)\}.\end{array}$$

Proof

Recall that F_e and f_e are the distribution and density functions of e_i,j. The assumption (e_i,j) = 0 implies that F_e(0) = 1/2. Notice that

$$\begin{array}{l}{J}_{\mu }(0,x)-L_{\mu }(x)/2=\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{b_n}(x_{i,j}-x)[\mathbb{P}\{Y_{i,j}\le \mu (x)\}-1/2]\\ =\sum _{i=1}^n\sum _{j=1}^{m_i}{K}_{b_n}(x_{i,j}-x)g(x,x_{i,j}),\end{array}$$

where g(x, v) = F_e{[μ(x)−μ(v)]/s(v)}−F_e(0). The symmetry of K entails ∫ u^sK(u)du = 0, s = 1, 3. The first expression then follows from Lemma 7.1 (ii) with r = 4.

Similarly, we can show $J_{\mu }^{\prime }(0,x):=\partial J_{\mu }({\delta }_1,x)/\partial {\delta }_1{\mid }_{{\delta }_1=0}=N_n{b}_n{f}_e(0)/[(b-a)s(x)]+O(1+N_n{b}_n^3)$ and $J_{\mu }^{″}({\delta }_1,x):={\partial }^2{J}_{\mu }({\delta }_1,x)/\partial {\delta }_1^2=O({Nb}_n)$ uniformly over δ₁, x. So, the second expression follows from the Taylor expansion $J_{\mu }({\delta }_1,x)-J_{\mu }(0,x)={\delta }_1{J}_{\mu }^{\prime }(0,x)+O({Nb}_n{\delta }_1^2)$. The other two expressions can be similarly treated. We omit the details.

7.4. Proof of Theorems 4.1–4.2

Let L_μ(x), L_μ(δ₁, x), J_μ(δ₁, x), L_s(x), L_s(δ₁, δ₂, x) and J_s(δ₁, δ₂, x) be as in Section 7.3.

Proof of Theorem 4.1

Let ${\delta }_n={[{(log{N}_n)}^3/(N_n{b}_n)]}^{1/2}+b_n^2\to 0$. Let l_n ↑ ∞ be a positive sequence satisfying δ_nl_n → 0. First, we show Δ̂_μ(x):= μ̂ (x) − μ(x) = O_p(l_nδ_n) uniformly over x ∈ ([a, b]). Since μ̂(x) is a solution to (4.1), by Koenker (2005, p.32–33),

$$\mid L_{\mu }({\widehat{\mathrm{\Delta }}}_{\mu }(x),x)-L_{\mu }(x)/2\mid \le \sum _{i,j}{K}_{b_n}(x_{i,j}-x){\mathbf{1}}_{Y_{i,j=\widehat{\mu }(x)}}=O_{\text{p}}(1),$$ (7.23)

uniformly over x. Let

$${\mathrm{\Omega }}_n(x)=[L_{\mu }(l_n{\delta }_n,x)-J_{\mu }(l_n{\delta }_n,x)]-[L_{\mu }(0,x)-J_{\mu }(0,x)].$$

We can apply Theorem 3.3 with ϖ_i,j (x) = K_bn(x_i,j − x) to Ω_n(x). For τ_n and φ_n in Condition 3.2, τ_n = O(1/b_n) and φ_n = O(N_nb_n) (see Lemma 7.1). By Theorem 3.3, sup_x∈[a,b] |Ω_n(x)| = O_p{[N_nb_nl_nδ_n(log N_n)³]^1/2}. By the same argument, we can show

$$\underset{x\in [a,b]}{sup}\mid L_{\mu }(0,x)-J_{\mu }(0,x)\mid =O_{\text{p}}\{{[N_n{b}_n{(log{N}_n)}^3]}^{1/2}\}.$$ (7.24)

Hence, by (7.24) and Lemma 7.2, uniformly over x ∈ ([a, b]),

$$\begin{array}{l}{L}_{\mu }(l_n{\delta }_n,x)-L_{\mu }(x)/2=[J_{\mu }(l_n{\delta }_n,x)-J_{\mu }(0,x)]+[J_{\mu }(0,x)-L_{\mu }(x)/2]+[L_{\mu }(0,x)-J_{\mu }(0,x)]+{\mathrm{\Omega }}_n(x)\\ =N_n{b}_n{l}_n{\delta }_n{f}_e(0)/[(b-a)s(x)][1+o(1)]+O_{\text{p}}({\nu }_n),\end{array}$$ (7.25)

where ${\nu }_n=N_n{b}_n^3+1+{[N_n{b}_n{(log{N}_n)}^3]}^{1/2}+{[N_n{b}_n{l}_n{\delta }_n{(log{N}_n)}^3]}^{1/2}$. Because l_n → ∞ and l_nδ_n → 0, it is easy to see that ν_n = o(N_nb_nl_nδ_n) and N_nb_nl_nδ_n → ∞, which implies L_μ(l_nδ_n, x) − L_μ(x)/2 → ∞ uniformly over x ∈ [a, b] in view of sup_x s(x) < ∞. Since L_μ(δ₁, x) is non-decreasing in δ₁, (7.23) and (7.25) entail ℙ{sup_x Δ̂_μ(x) ≤ l_nδ_n} → 1. Similarly, ℙ{inf_x Δ̂_μ(x) ≥ −l_nδ_n} → 1. So, sup_x | Δ̂_μ(x)| = O_p(l_nδ_n). Since the rate of l_n → ∞ can be arbitrarily slow, sup_x |Δ̂_μ(x)| = O_p(δ_n).

Again, by (7.23) and Lemma 7.2, uniformly over x ∈ ([a, b]),

$$\begin{array}{l}{L}_{\mu }(0,x)-J_{\mu }(0,x)=L_{\mu }({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),x)-J_{\mu }({\widehat{\mathrm{\Delta }}}_{\mu }(x),x)+O_{\text{p}}[\sqrt{N_n{b}_n{\delta }_n{(log{N}_n)}^3}]\\ =[L_{\mu }({\widehat{\mathrm{\Delta }}}_{\mu }(x),x)-L_{\mu }(x)/2]+[L_{\mu }(x)/2-J_{\mu }(0,x)]-[J_{\mu }({\widehat{\mathrm{\Delta }}}_{\mu }(x),x)-J_{\mu }(0,x)]+O_{\text{p}}[\sqrt{N_n{b}_n{\delta }_n{(log{N}_n)}^3}]\\ =O_{\text{p}}(1)+N_n{b}_n^3{\rho }_{\mu }(x)f_e(0){\psi }_K/[(b-a)s(x)]+O(1+N_n{b}_n^5)-N_n{b}_n{\widehat{\mathrm{\Delta }}}_{\mu }(x)\{f_e(0)/[(b-a)s(x)]+O({\delta }_n)\}+O_{\text{p}}[\sqrt{N_n{b}_n{\delta }_n{(log{N}_n)}^3}].\end{array}$$

The representation (4.8) then follows by solving Δ̂_μ(x) from the above equation.

Proof of Theorem 4.2

We use the argument in Theorem 4.1 and only sketch the outline. Let

$$D_s({\delta }_1,{\delta }_2,x)=[L_s({\delta }_1,{\delta }_2,x)-J_s({\delta }_1,{\delta }_2,x)]-[L_s(0,0,x)-J_s(0,0,x)].$$

Using Theorem 3.3, we can show that

$$\underset{x\in [a,b]}{sup}\mid L_s(0,0,x)-J_s(0,0,x)\mid =O_{\text{p}}\{{[N_n{h}_n{(log{N}_n)}^3]}^{1/2}\},$$ (7.26)

$$\underset{\mid {\delta }_1\mid +\mid {\delta }_2\mid \le {\delta }_n,x\in [a,b]}{sup}\mid D_s({\delta }_1,{\delta }_2,x)\mid =O_{\text{p}}\{{[N_n{h}_n{(log{N}_n)}^3]}^{1/2}\},$$ (7.27)

hold for all b_n → 0, h_n → 0 and δ_n → 0 satisfying sup_n log N_n/[N_n min(b_n, h_n)δ_n] < ∞.

Let ${\delta }_n=b_n^2+h_n^2+{[{(log{N}_n)}^3/(N_n{b}_n)]}^{1/2}+{[{(log{N}_n)}^3/(N_n{h}_n)]}^{1/2}$ and l_n → ∞ be a sequence such that l_nδ_n → 0. By Theorem 4.1, Δ̃_μ(x):= μ̃(x) − μ(x) = O_p(δ_n). Using (7.27) and Lemma 7.2, we can derive the following counterpart of (7.25)

$$\begin{array}{l}{L}_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),l_n{\delta }_n,x)-L_s(x)/2=[J_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),l_n{\delta }_n,x)-J_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),0,x)]+[J_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),0,x)-L_s(x)/2]+L_s(0,0,x)-J_s(0,0,x)+O_{\text{p}}\{{[N_n{h}_n{l}_n{\delta }_n{(log{N}_n)}^3]}^{1/2}\}\\ =N_n{h}_n{l}_n{\delta }_n{\kappa }_{+}/[(b-a)s(x)][1+o_{\text{p}}(1)]\to \infty .\end{array}$$

Let Δ̂_s(x) = ŝ(x) − s(x). By the same argument in (7.23), sup_x |L_s(Δ̃_μ(x), Δ̂_s(x), x) −L_s(x)/2| = O_p(1). Notice that L_s(Δ̃_μ(x), δ₂, x) is non-decreasing in x. Thus, ℙ{sup_x Δ̂_s(x) ≤ l_nk_n} → 1. Similarly, ℙ{inf_x Δ̂_s(x) ≥ −l_nk_n} → 1. Then sup_x | Δ̂_s(x)| = O_p(δ_n).

Write ϖ_n = [N_nh_nδ_n(log N_n)³]^1/2. To derive the Bahadur representation (4.14), we use (7.27) and Lemma 7.2 to obtain

$$\begin{array}{l}{L}_s(0,0,x)-J_s(0,0,x)=[L_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),{\widehat{\mathrm{\Delta }}}_s(x),x)-L_s(x)/2]+[L_s(x)/2-J_s({\widehat{\mathrm{\Delta }}}_{\mu }(x),0,x)]-[J_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),{\widehat{\mathrm{\Delta }}}_s(x),x)-J_s({\stackrel{\sim }{\mathrm{\Delta }}}_{\mu }(x),0,x)]+O_{\text{p}}({\varpi }_n)\\ =O_{\text{p}}(1)+N_n{h}_n{\kappa }_{+}\{[h_n^2{\psi }_K{\rho }_s(x)-\kappa {\stackrel{\sim }{\mathrm{\Delta }}}_u(x)]/[(b-a)s(x)]+O(h_n^4+{\delta }_n^2)\}-N_n{h}_n{\widehat{\mathrm{\Delta }}}_s(x)\{{\kappa }_{+}/[(b-a)s(x)]+O({\delta }_n)\}+O_{\text{p}}({\varpi }_n).\end{array}$$

Solving Δ̂_s(x) from the above equation, we obtain the Bahadur representation (4.14).

7.5. Proof of Corollaries 4.1–4.2

Again we use the coupling argument to convert the dependent data to m-dependent case. Theorem 7.1 below presented a CLT for m-dependent sequence with unbounded m.

Theorem 7.1 (Romano and Wolf (2000))

Let Z_n,j, 1 ≤ j ≤ d_n, be a triangular array of mean zero k_n-dependent random variables. Define

$$S_n=\sum _{j=1}^{d_n}{Z}_{n,j},B_n^2=\text{Var}(S_n),S_{n,h,a}=\sum _{j=a}^{a+h-1}{Z}_{n,j},B_{n,h,a}^2=\text{Var}(S_{n,h,a}).$$

Assume that there exist some δ > 0, −1 ≤ γ < 1, C_n,1, C_n,2, C_n,3 > 0 such that

1. (|Z_n,j|^2+δ) = O(C_n,1);

2. (b) $B_{n,h,a}^2/h^{1+\gamma }=O(C_{n,2})$ for all h ≥ k_n,a;

3. (c) $B_n^2/(d_n{C}_{n,2}^{\gamma })\ge C_{n,3}$;

4. C_n,2/C_n,3 = O(1);

5. $C_{n,1}/C_{n,3}^{(2+\delta )/2}=O(1)$;

6. $k_n^{1+(1-\gamma )(1+2/\delta )}/d_n\to 0$.

Then S_n/B_n ⇒ N(0, 1).

Proof of Corollaries 4.1–4.2

We only prove Corollary 4.1 since Corollary 4.2 can be similarly treated. By the Bahadur representation (4.8), under the specified condition, $r_n\sqrt{N_n{b}_n}\to 0$. Thus, it suffices to show (N_nb_n)^−1/2Q_bn (x) ⇒ N (0, ϕ_K/[4(b − a)]). Recall e_i,j(k_n) and Ỹ_i,j in (3.1) and (3.5). Define the coupling process

$${\stackrel{\sim }{Q}}_{b_n}(x)=-\sum _{i=1}^n\sum _{j=1}^{m_i}\{{\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)}-\mathbb{E}[{\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)}]\}{K}_{b_n}(x_{i,j}-x).$$

Let the coupling lag k_n = ⌊c log N_n⌋ be chosen as in Theorem 3.2. By Theorem 3.2, Q_bn (x)−Q̃_bn (x) = O_p[(log N_n)²] = o_p[(N_nb_n)^1/2]. It remains to show (N_nb_n)^−1/2Q̃_bn (x) ⇒ N(0, ϕ_K/[4(b − a)]). Recall M_n = max_1≤i≤n m_i. Set Ỹ_i,j = 0 for m_i < j ≤ M_n. Define

$$Z_{n,j}=\sum _{i=1}^n{\zeta }_{i,j},\text{where}{\zeta }_{i,j}={(N_n{b}_n)}^{-1/2}\{{\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)}-\mathbb{E}[{\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)}]\}{K}_{b_n}(x_{i,j}-x).$$

Then we can write $-{(N_n{b}_n)}^{-1/2}{\stackrel{\sim }{Q}}_{b_n}(x)={\sum }_{j=1}^{M_n}{Z}_{n,j}$. Notice that Z_n,j, j = 1, 2, …, are (2k_n + 1)-dependent, and ζ_i,j, i = 1, 2, …, are independent for each fixed j.

Let S_n, $B_n^2$, S_n,h,a and $B_{n,h,a}^2$ be defined in Theorem 7.1. We shall verify the conditions in Theorem 7.1. By the independence of the summands ζ_i,j in Z_n,j,

$$\begin{array}{l}\mathbb{E}({\mid Z_{n,j}\mid }^4)=\sum _{i=1}^n\mathbb{E}({\mid {\zeta }_{i,j}\mid }^4)+6\sum _{i_1\ne i_2}\mathbb{E}({\mid {\zeta }_{i,j}\mid }^2)\mathbb{E}({\mid {\zeta }_{i_2,j}\mid }^2)\\ =\frac{O(1)}{{(N_n{b}_n)}^2}\left\{\sum _{i=1}^n{K}_{b_n}^4(x_{i,j}-x)+{\left[\sum _{i=1}^n{K}_{b_n}^2(x_{i,j}-x)\right]}^2\right\}=O(1/M_n^2),\end{array}$$

in view of nM_n = O(N_n). Since Ỹ_i,j and Y_i,j have same distribution, we have $g(x,x_{i,j}):=\text{Var}({\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)})=F_e\{[\mu (x)-\mu (x_{i,j})]/s(x_{i,j})\}-F_e^2\{[\mu (x)-\mu (x_{i,j})]/s(x_{i,j})\}$. Recall F_e(0) = 1/2. Then g(x, x) = 1/4. Thus, by (4.9) and the (2k_n + 1)-dependence of Ỹ_i,j, j ∈ , applying Lemma 7.2 (ii) with r = 1 produces

$$\begin{array}{l}{B}_n^2=\frac{1}{N_n{b}_n}\sum _{i=1}^n\sum _{j=1}^{n_i}\text{Var}({\mathbf{1}}_{{\stackrel{\sim }{Y}}_{i,j}\le \mu (x)})K_{b_n}^2(x_{i,j}-x)+\frac{O(1)}{N_n{b}_n}\sum _{i=1}^n\sum _{1\le j_1<j_2\le n_i,\mid j_1-j_2\mid \le 2k_n}{K}_{b_n}(x_{i,j_1}-x)K_{b_n}(x_{i,j_2}-x)\\ =\frac{1}{N_n{b}_n}\left[\frac{N_n{b}_n{\varphi }_K}{4(b-a)}+O(N_n{b}_n^2)\right]+\frac{O({nM}_n{k}_n{b}_n{t}_n)}{N_n{b}_n}\to \frac{{\varphi }_K}{4(b-a)},\end{array}$$

in view of nM_n = O(N_n) and k_nι_n → 0. Similarly, we can show $B_{n,h,a}^2=O(nh/N_n)=O(h/M_n)$. Therefore, it is easy to see that the conditions in Theorem 7.1 hold with δ = 2, γ = 0, and straightforward choices of C_n,1, C_n,2, C_n,3, completing the proof.