Bahadur representations of M-estimators and their applications in general linear models

Abstract

Consider the linear regression model

$$y_i=x_i^T\beta +e_i,i=1,2,\dots ,n,$$

where $e_i=g(\dots ,{\epsilon }_{i-1},{\epsilon }_i)$ are general dependence errors. The Bahadur representations of M-estimators of the parameter β are given, by which asymptotically the theory of M-estimation in linear regression models is unified. As applications, the normal distributions and the rates of strong convergence are investigated, while $\{{\epsilon }_i,i\in Z\}$ are m-dependent, and the martingale difference and $(\epsilon ,\psi )$-weakly dependent.

Introduction

Consider the following linear regression model:

$$y_i=x_i^T\beta +e_i,i=1,2,\dots ,n,$$ 1.1

where $\beta ={({\beta }_1,\dots ,{\beta }_p)}^T\in R^p$ is an unknown parametric vector, $x_i^T$ denotes the ith row of an $n\times p$ design matrix X, and $\{e_i\}$ are stationary dependence errors with a common distribution.

An M-estimate of β is defined as any value of β minimizing

$$\sum _{i=1}^n\rho (y_i-x_i^T\beta )$$ 1.2

for a suitable choice of the function ρ, or any solution for β of the estimating equation

$$\sum _{i=1}^n\psi (y_i-x_i^T\beta )x_i=0$$ 1.3

for a suitable choice of ψ.

There is a body of statistical literature dealing with linear regression models with independent and identically distributed (i.i.d.) random errors, see e.g. Babu [1], Bai et al. [2], Chen [7], Chen and Zhao [8], He and Shao [24], Gervini and Yohai [23], Huber and Ronchetti [28], Xiong and Joseph [50], Salibian-Barrera et al. [44]. Recently, linear regression models with serially correlated errors have attracted increasing attention from statisticians; see, for example, Li [33], Wu [49], Maller [38], Pere [41], Hu [25, 26]. Over the last 40 years, M-estimators in linear regression models have been investigated by many authors. Let $\{{\eta }_i\}$ be i.i.d. random variables. Koul [30] discussed the asymptotic behavior of a class of M-estimators in the model (1.1) with long range dependence errors $e_i=G({\eta }_i)$. Wu [49] and Zhou and Shao [52] discussed the model (1.1) with $e_i=G(\dots ,{\eta }_{i-1},{\eta }_i)$ and derived strong Bahadur representations of M-estimators and a central limit theorem. Zhou and Wu [53] considered the model (1.1) with $e_i={\sum }_{j=0}^{\mathrm{\infty }}{a}_j{\eta }_{i-j}$, and obtained some asymptotic results including consistency of robust estimates. Fan et al. [20] investigated the model (1.1) with the errors $e_i=f(e_{i-1})+{\eta }_i$ and established the moderate deviations and strong Bahadur representations for M-estimators. Wu [47] discussed strong consistency of an M-estimator in the model (1.1) for negatively associated samples. Fan [19] considered the model (1.1) with φ-mixing errors, and the moderate deviations for the M-estimators. In addition, Berlinet et al. [4], Boente and Fraiman [5], Chen et al. [6], Cheng et al. [9], Gannaz [22], Lô and Ronchetti [37], Valdora and Yohai [45] and Yang [51] have also studied some asymptotic properties of M-estimators in nonlinear models. However, no people have investigated a unified the theory of M-estimation in linear regression models with more general errors.

In this paper, we assume that

$$e_i=g(\dots ,{\epsilon }_{i-1},{\epsilon }_i),$$ 1.4

where $g(\cdot )$ is a measurable function such that $e_i$ is a proper random variable, and $\{{\epsilon }_i,i\in Z\}$ (where Z is the set of integers) are very general random variables, including m-dependent, martingale difference, $(\epsilon ,\psi )$-weakly dependent, and so on.

We try to investigate the unified the theory of M-estimation in the linear regression model. In the article, we use the idea of Wu [49] to study the Bahadur representative of M-estimator, and we extend some results to general errors. The paper is organized as follows. In Sect. 2, the weak and strong linear representation of an M-estimate of the vector regression parameter β in the model (1.1) are presented. Section 3 contains some applications of our results, including the m-dependent, $(\epsilon ,\psi )$-weakly dependent, martingale difference. In Sect. 4, proofs of the main results are given.

Main results

In the section, we investigate the weak and strong linear representation of an M-estimate of the vector regression parameter β in the model (1.1). Without loss of generality, we assume that the true parameter $\beta =0$. We start with some notation and assumptions.

For a vector $v=(v_1,\dots ,v_p)$, let $|v|={({\sum }_{i=1}^p{v_i}^2)}^{\frac{1}{2}}$. A random vector V is said to be in $L^q,q>0$, if $E({|V|}^q)<\mathrm{\infty }$. Let ${\parallel V\parallel }_q=E{({|V|}^q)}^{\frac{1}{q}}$, $\parallel V\parallel ={\parallel V\parallel }_2$, ${\mathrm{\Sigma }}_n={\sum }_{i=1}^n{x}_i{x}_i^T=X^{T}X$ and assume that ${\mathrm{\Sigma }}_n$ is positive definite for large enough n. Let $x_{in}={\mathrm{\Sigma }}_n^{-\frac{1}{2}}{x}_i,{\beta }_n={\mathrm{\Sigma }}_n^{-\frac{1}{2}}\beta$. Then the model (1.1) can be written as

$$y_i=x_{in}^T{\beta }_n+e_i,i=1,2,\dots ,n,$$ 2.1

with ${\sum }_{i=1}^n{x}_{in}{x}_{in}^T=I_p$, where $I_p$ is an identity matrix of order p. Assume that ρ has derivative ψ. For $l\ge 0$ and a function f, write $f\in C^l$ if f has derivatives up to lth order and $f^{(l)}$ is continuous. Define the function

$${\psi }_k(t;{\mathcal{F}}_i)=E(\psi (e_k+t)|{\mathcal{F}}_i),{\psi }_k(t;{\mathcal{F}}_i^{\ast })=E(\psi (e_k^{\ast }+t)|{\mathcal{F}}_i^{\ast }),k\ge 0,$$ 2.2

where ${\mathcal{F}}_i^{\ast }=(\dots ,{\epsilon }_{-1},{\epsilon }_0^{\prime },{\epsilon }_1,\dots ,{\epsilon }_{i-1},{\epsilon }_i),{\mathcal{F}}_i=(\dots ,{\epsilon }_{-1},{\epsilon }_0,{\epsilon }_1,\dots ,{\epsilon }_{i-1},{\epsilon }_i)$, let ${\epsilon }_i^{\prime }$ be an i.i.d. copy of ${\epsilon }_i$, and $e_k^{\ast }=g({\mathcal{F}}_k^{\ast })$.

Throughout the paper, we use the following assumptions.

1. $\rho (\cdot )$ is a convex function, $E\psi (e_i)=0,0<E{\psi }^2(e_i)$.

2. $\phi (t)\equiv E\psi (e_i+t)$ has a strictly positive derivative at $t=0$.

3. $m(t)\equiv E{|\psi (e_i+t)-\psi (e_i)|}^2$ is continuous at $t=0$.

4. $r_n\equiv {max}_{1\le i\le n}|x_{in}|={max}_{1\le i\le n}{(x_i^T{\mathrm{\Sigma }}_n^{-1}{x}_i)}^{\frac{1}{2}}=o(1)$.

5. There exists a ${\delta }_0>0$ such that

   $$L_i\equiv \underset{|s|,|t|\le {\delta }_0,s\ne t}{sup}\frac{|{\psi }_{i+1}(s;{\mathcal{F}}_i)-{\psi }_{i+1}(t;{\mathcal{F}}_i)|}{|s-t|}\in L^1.$$ 2.3
6. Let ${\psi }_i(\cdot ;{\mathcal{F}}_{i-1})\in C^l,l\ge 0$. For some ${\delta }_0>0,{max}_{1\le i\le n}{sup}_{|\delta |\le {\delta }_0}\parallel {\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1})\parallel <\mathrm{\infty }$ and

   $$\sum _{i=0}^{\mathrm{\infty }}\underset{|\delta |<{\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1})|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0^{\ast })\parallel <\mathrm{\infty }.$$ 2.4
7. $$\sum _{i=0}^{\mathrm{\infty }}\underset{|\delta |<{\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_{-1})\parallel <\mathrm{\infty },$$ 2.5

   $$\sum _{i=0}^{\mathrm{\infty }}\underset{|\delta |<{\delta }_0}{sup}|E{\psi }^{(l)}(e_i+\delta )-E{\psi }^{(l)}(e_i^{\ast }+\delta )|<\mathrm{\infty }.$$ 2.6

Remark 1

Conditions (A1)–(A5) and (A6) are imposed in the M-estimation considering the theory of linear regression models with dependent errors (Wu [49]; Zhou and Shao [52]). Condition (2.6) is similar to (7) of Wu [49]. $\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1})|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0^{\ast })\parallel$ measures the difference of the contribution of ${\epsilon }_0$ and its copy ${\epsilon }_0^{\prime }$ in predicting $\psi (e_i+\delta )$. However, $\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_{-1})\parallel$ measures the contribution of ${\epsilon }_0$ in predicting $\psi (e_i+\delta )$ under the given copy of ${\epsilon }_0$: ${\epsilon }_0^{\prime }$.

If $\{{\epsilon }_i\}$ are i.i.d., then (A6) and (A7) hold. For the other settings, (A6) and (A7) are very easily satisfied. The following proposition provides some sufficient conditions for (A6) and (A7).

Proposition 2.1

Let $F_i(u|{\mathcal{F}}_0)=P(e_i\le u|{\mathcal{F}}_0)$ and $f_i(u|{\mathcal{F}}_0)$ be the conditional distribution and density function of $e_i$ at u given ${\mathcal{F}}_0$, respectively. Let $f_i(u)$ and $f_i^{\ast }(u)$ be the density function of $e_i$ and $e_i^{\ast }$, respectively.

1. Let $f_i(\cdot |{\mathcal{F}}_i)\in C^l,l\ge 0$, $\omega (i)={\int }_R\parallel f_i(u|{\mathcal{F}}_0)-f_i(u|{\mathcal{F}}_0^{\ast })\parallel \psi (u;{\delta }_0)du$ and $\psi (u;{\delta }_0)=|\psi (u+{\delta }_0)|+|\psi (u-{\delta }_0)|$. If ${\sum }_{i=1}^{\mathrm{\infty }}\omega (i)<\mathrm{\infty }$, then (A6) holds.

2. Let

   $$\bar{\omega }(i)={\int }_R\parallel f_i^{(l)}(u|{\mathcal{F}}_0)-f_i^{(l)}(u|{\mathcal{F}}_0^{\ast })\parallel \psi (u;{\delta }_0)du$$

   and $\tilde{\omega }(i)={\int }_R|f_i(u)-f_i^{\ast }(u)|{\psi }^{(l)}(u;{\delta }_0)du$. If ${\sum }_{i=1}^{\mathrm{\infty }}\bar{\omega }(i)<\mathrm{\infty }$ and ${\sum }_{i=1}^{\mathrm{\infty }}\tilde{\omega }(i)<\mathrm{\infty }$, then assumption (A7) holds.

Proof

(1) By the conditions of (1), we have

$$\begin{array}{rl}& \sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1})|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0^{\ast })\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel {\int }_R{\psi }^{(l)}(u+\delta )[f_i(u|{\mathcal{F}}_0)-f_i(u|{\mathcal{F}}_0^{\ast })]du\parallel \\ & +\underset{|\delta |\le {\delta }_0}{sup}\parallel {\int }_R{\psi }^{(l)}(u+\delta )[f_0(u|{\mathcal{F}}_{-1})-f_0(u|{\mathcal{F}}_{-1})]du\parallel \\ & \le \sum _{i=1}^{\mathrm{\infty }}{\int }_R\parallel f_i^{(l)}(u|{\mathcal{F}}_0)-f_i^{(l)}(u|{\mathcal{F}}_0^{\ast })\parallel \psi (u+{\delta }_0)du\\ & =\sum _{i=1}^{\mathrm{\infty }}\omega (i)<\mathrm{\infty }.\end{array}$$ 2.7

Namely (A6) holds.

(2) (A7) follows from

$$\begin{array}{rl}& \sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_{-1})\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel {\int }_R{\psi }^{(l)}(u+\delta )[f_i^{(l)}(u|{\mathcal{F}}_0)-f_i^{(l)}(u|{\mathcal{F}}_{-1})]du\parallel \\ & +\underset{|\delta |\le {\delta }_0}{sup}\parallel {\int }_R{\psi }^{(l)}(u+\delta )[f_0(u|{\mathcal{F}}_{-1})-f_0(u|{\mathcal{F}}_{-1}^{\ast })]du\parallel \\ & \le \sum _{i=1}^{\mathrm{\infty }}{\int }_R\parallel f_i^{(l)}(u|{\mathcal{F}}_0)-f_i^{(l)}(u|{\mathcal{F}}_{-1})\parallel \psi (u+{\delta }_0)du=\sum _{i=1}^{\mathrm{\infty }}\bar{\omega }(i)<\mathrm{\infty }\end{array}$$

and

$$\begin{array}{rl}& \sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}|E({\psi }^{(l)}(e_i+\delta )-E({\psi }^{(l)}(e_i^{\ast }+\delta )|\\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}|{\int }_R{\psi }^{(l)}(u+\delta )(f_i(u)-f_i^{\ast }(u))|\\ & \le \sum _{i=1}^{\mathrm{\infty }}{\int }_R|f_i(u)-f_i^{\ast }(u)|{\psi }^{(l)}(u;{\delta }_0)du=\sum _{i=1}^{\mathrm{\infty }}\tilde{\omega }(i)<\mathrm{\infty }.\end{array}$$

Hence, the proposition is proved. □

Define the M-processes

$$K_n({\beta }_n)={\mathrm{\Omega }}_n({\beta }_n)-E({\mathrm{\Omega }}_n({\beta }_n)),{\tilde{K}}_n(\beta )={\tilde{\mathrm{\Omega }}}_n(\beta )-E({\tilde{\mathrm{\Omega }}}_n(\beta )),$$

where

$${\mathrm{\Omega }}_n({\beta }_n)=\sum _{i=1}^n\psi (e_i-x_{in}^T{\beta }_n)x_{in},{\tilde{\mathrm{\Omega }}}_n(\beta )=\sum _{i=1}^n\psi (e_i-x_i^T\beta )x_i.$$

Theorem 2.1

Let $\{{\delta }_n,n\in N\}$ be a sequence of positive numbers such that ${\delta }_n\to \mathrm{\infty }$ and ${\delta }_n{r}_n\to 0$. If (A1)–(A5), and (A6) and (A7) with $l=0,1,\dots ,p$ hold, then

$$\underset{|\beta |\le {\delta }_n}{sup}|K_n({\beta }_n)-K_n(0)|=O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}),$$ 2.8

where

$${\tau }_n(\delta )=\sum _{i=1}^n|x_{in}{|}^2(m^2(|x_{in}|\delta )+m^2(-|x_{in}|\delta )),\delta >0.$$

Corollary 2.1

Assume that (A1)–(A5), and (A6) and (A7) with $l=0,1,\dots ,p$ hold. If $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ as $t\to 0$, $\mathrm{\Omega }({\hat{\beta }}_n)=O_p(r_n)$, then, for $|{\hat{\beta }}_n|\le {\delta }_n$,

$${\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}=O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n^2{r}_n).$$ 2.9

Moreover, if, as $t\to 0$, $m(t)=O({|t|}^{\lambda })$ for some $\lambda >0$, then

$${\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}=O_p(\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2\lambda }}logn+r_n).$$ 2.10

Remark 2

If $\{e_i\}$ i.i.d., then $|{\hat{\beta }}_n|\le {\delta }_n$ follows from (3.2) of Rao and Zhao [42]. If $\{{\epsilon }_i\}$ i.i.d., then $|{\hat{\beta }}_n|\le {\delta }_n$ follows from Theorem 1 of Wu [49] and Zhou and Shao [52]. If $e_i=f(e_{i-1})+{\epsilon }_i$, where the function $f:R\times R\to R$ satisfies some condition and $\{{\epsilon }_i\}$ i.i.d., then $|{\hat{\beta }}_n|\le {\delta }_n$ follows from Theorem 2.2 of Fan et al. [20]. If $\{{\epsilon }_i\}$ NA, then $|{\hat{\beta }}_n|\le {\delta }_n$ follows from Theorem 1 of Wu [47]. Therefore the condition $|{\hat{\beta }}_n|\le {\delta }_n$ is not strong. In the paper, we do not discuss it.

Theorem 2.2

Assume that (A1)–(A3), (A5), and (A6) and (A7) with $l=0,1,\dots ,p$ hold. Let ${\lambda }_n$ be the minimum eigenvalue of ${\mathrm{\Sigma }}_n$, $b_n=n^{-\frac{1}{2}}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }$, $\upsilon >0$, $\tilde{n}=2^{\lceil logn/log2\rceil }$ and $q>\frac{3}{2}$. If ${lim\hspace{0.17em}inf}_{n\to \mathrm{\infty }}{\lambda }_n/n>0,{\sum }_{i=1}^n{|x_i|}^2=O(n)$ and ${\tilde{r}}_n={max}_{1\le i\le n}|x_i|=O(n^{1/2}{(logn)}^{-2})$, then

$$\underset{|\beta |\le b_n}{sup}|{\tilde{K}}_n(\beta )-{\tilde{K}}_n(0)|=O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}}),$$

where $L_n=\sqrt{{\tilde{\tau }}_n(2b_n)}{({log}_n)}^q$, $B_{\tilde{n}}=b_n{({\sum }_{i=1}^n{|x_i|}^4)}^{1/2}{(logn)}^{3/2}{(loglogn)}^{(1+\upsilon )/2}$ and

$${\tilde{\tau }}_n(\delta )=\sum _{i=1}^n|x_i{|}^2(m^2(|x_i|\delta )+m^2(-|x_i|\delta )),\delta >0.$$

Corollary 2.2

Assume that $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ and $m(t)=O(\sqrt{t})$ as $t\to 0$, and ${\tilde{\mathrm{\Omega }}}_n=O_{a.s.}({\tilde{r}}_n)$. Under the conditions of Theorem 2.2, we have:

1. ${\tilde{\beta }}_n=O_{a.s.}(b_n)$;

2. ${\phi }^{\prime }(0){\mathrm{\Sigma }}_n{\tilde{\beta }}_n-{\sum }_{i=1}^n\psi (e_i)x_i=O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}}+b_n^2{\sum }_{i=1}^n{|x_i|}^3+{\tilde{r}}_n)$,

where ${\tilde{\beta }}_n$ is the minimizer of (1.2).

Remark 3

From the above results, we easily obtain the corresponding conclusions of Wu [49].

From the corollary below, we only derive convergence rates of ${\tilde{\beta }}_n$. However, it is to be regretted that we cannot give laws of the iterated logarithm $n^{1/2}{(loglogn)}^{1/2}$, which is still an open problem.

Corollary 2.3

Under the conditions of Corollary 2.2, we have

$$\begin{array}{rl}{\mathrm{\Sigma }}_n{\tilde{\beta }}_n=& O_{a.s.}(max\{\left(n^{1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }\right),\\ & \left(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+\upsilon /2}\right),(\sum _{i=1}^n\psi (e_i)x_i)\left\}\right).\end{array}$$

Proof

Note that $\tilde{n}=2^{\lceil logn/log2\rceil }=O(n)$ and $m(t)=O(\sqrt{t})$ as $t\to 0$; we have

$$\begin{array}{rl}{L}_{\tilde{n}}& =\sqrt{{\tilde{\tau }}_n(2b_n)}{(logn)}^q=\sqrt{O(\sum _{i=1}^n|x_i{|}^2|x_i|b_n)}{(logn)}^q\\ & =\sqrt{O\left(n{n}^{1/2}{(logn)}^{-2}{n}^{-1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }\right)}{(logn)}^q\\ & =O\left(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+\upsilon /2}\right),\\ B_{\tilde{n}}& =O\left(n^{-1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }{\left(n{\tilde{r}}_n^2\right)}^{1/2}{(logn)}^{3/2}{(loglogn)}^{(1+\upsilon )/2}\right)\\ & =O(n^{1/2}(logn){(loglogn)}^{1+3\upsilon /2})\end{array}$$

and

$$\begin{array}{rl}{b}_n^2\sum _{i=1}^n|x_i{|}^3& =O\left(n^{-1}{(logn)}^3{(loglogn)}^{1+2\upsilon }n{n}^{1/2}{(logn)}^{-2}\right)\\ & =O(n^{1/2}(logn){(loglogn)}^{1+2\upsilon }).\end{array}$$

By Corollary 2.2, we have

$${\phi }^{\prime }(0){\mathrm{\Sigma }}_n{\tilde{\beta }}_n=\sum _{i=1}^n\psi (e_i)x_i+O_{a.s.}\left(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+\upsilon /2}\right)$$ 2.11

and

$${\mathrm{\Sigma }}_n{\tilde{\beta }}_n=O_{a.s.}(n{b}_n)=O_{a.s.}\left(n^{1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }\right).$$ 2.12

Thus the conclusion follows from (2.11) and (2.12). □

Applications

In the following three subsections, we shall investigate some applications of our results. In Sect. 3.1, we consider that ${\epsilon }_i$ is a m-dependent random variable sequence. We shall investigate that $\{{\epsilon }_i\}$ are $(\epsilon ,\psi )$-weakly dependent in Sect. 3.2, and martingale difference errors $\{{\epsilon }_i\}$ in Sect. 3.3.

m-dependent process

In the subsection, we shall firstly show that the m-dependent sequence satisfies conditions (A6) and (A7) and secondly obtain the asymptotic normal distribution and strong convergence rates for M-estimators of the parameter. Koul [30] discussed the asymptotic behavior of a class of M-estimators in the model (1.1) with long range dependence errors $e_i=g({\epsilon }_i)$, where ${\epsilon }_i$ i.i.d. Here we assume that ${\epsilon }_i$ is a m-dependent sequence, of which the definition was given by Example 2.8.1 in Lehmann [32]. For m-dependent sequences or processes, there are some results (e.g., see Hu et al. [27], Romano and Wolf [43] and Valk [46]).

Proposition 3.1

Let ${\epsilon }_i$ in (1.4) be a m-dependent sequence. Then (A6) and (A7) hold.

Proof

Note that ${\epsilon }_i$ is a m-dependent sequence, we have

$$\begin{array}{rl}& \sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1})|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0^{\ast })\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }^{(l)}(e_i+\delta )|{\mathcal{F}}_0)-E({\psi }^{(l)}(e_i^{\ast }+\delta )|{\mathcal{F}}_0^{\ast })\parallel \\ & +\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }^{(l)}(e_i+\delta )|{\mathcal{F}}_{-1})-E({\psi }^{(l)}(e_i^{\ast }+\delta )|{\mathcal{F}}_{-1})\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }^{(l)}(e_i+\delta ))-E\left({\psi }^{(l)}(e_i^{\ast }+\delta )\right)\parallel \\ & =0<\mathrm{\infty }\end{array}$$ 3.1

and

$$\begin{array}{rl}& \sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_0)-E({\psi }_i^{(l)}(\delta ;{\mathcal{F}}_{i-1}^{\ast })|{\mathcal{F}}_{-1})\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }^{(l)}(e_i^{\ast }+\delta )|{\mathcal{F}}_0)-E({\psi }^{(l)}(e_i^{\ast }+\delta )|{\mathcal{F}}_{-1})\parallel \\ & +\underset{|\delta |\le {\delta }_0}{sup}\parallel E({\psi }^{(l)}(e_0+\delta )|{\mathcal{F}}_{-1})-E({\psi }^{(l)}(e_0^{\ast }+\delta )|{\mathcal{F}}_{-1})\parallel \\ & =\sum _{i=1}^{\mathrm{\infty }}\underset{|\delta |\le {\delta }_0}{sup}\parallel E{\psi }^{(l)}(e_0+\delta )-E{\psi }^{(l)}(e_i^{\ast }+\delta )\parallel \\ & =0<\mathrm{\infty }.\end{array}$$ 3.2

Therefore, (A6) and (A7) follow from (3.1), (3.2) and $E{\psi }^{(l)}(e_i+\delta )=E{\psi }^{(l)}(e_i^{\ast }+\delta )$. □

Corollary 3.1

Assume that (A1)–(A5) hold. If $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ and $m(t)=O({|t|}^{\lambda })$ for some $\lambda >0$ as $t\to 0$, $\mathrm{\Omega }({\hat{\beta }}_n)=0$ and $0<{\sigma }_{\psi }^2=E{[\psi (e_i)]}^2<\mathrm{\infty }$, then

$$n^{-1/2}{\hat{\beta }}_n/\left({[{\phi }^{\prime }(0)]}^{-1}{\sigma }_{\psi }\right)\to N(0,I_p),n\to \mathrm{\infty }.$$

In order to prove Corollary 3.1, we give the following lemmas.

Lemma 3.1

(Lehmann [32])

Let $\{{\xi }_i,i\ge 1\}$ be a stationary m-dependent sequence of random variables with $E{\xi }_i=0$ and $0<{\sigma }^2=Var({\xi }_i)<\mathrm{\infty }$, and $T_n={\sum }_{i=1}^n{\xi }_i$. Then

$$n^{-1/2}{T}_n/\tau \to N(0,1),$$

where ${\tau }^2={lim}_{n\to \mathrm{\infty }}Var(n^{-1/2}{T}_n)={\sigma }^2+2{\sum }_{i=2}^{m+1}Cov({\xi }_1,{\xi }_i)$.

Using the argument of Lemma 3.1, we easily obtain the following result. Here we omit the proof.

Lemma 3.2

Let $\{{\xi }_i,i\ge 1\}$ be a stationary m-dependent sequence of random variables with $E{\xi }_i=0$ and $0<{\sigma }_i^2=Var({\xi }_i)<\mathrm{\infty }$, and $T_n={\sum }_{i=1}^n{\xi }_i$. Then

$$n^{-1/2}{T}_n/\tau \to N(0,1),$$

where ${\tau }^2=Var(n^{-1/2}{T}_n)=n^{-1}{\sum }_{i=1}^n{\sigma }_i^2+2n^{-1}{\sum }_{i=2}^{m+1}(n-i)Cov({\xi }_1,{\xi }_i)$.

Proof of Corollary 3.1

By (2.10), we have

$$n^{-1/2}{\hat{\beta }}_n=n^{-1/2}{[{\phi }^{\prime }(0)]}^{-1}\sum _{i=1}^n\psi (e_i)x_{in}+O_p(n^{-1/2}{r}_n^{\lambda }logn).$$ 3.3

Since $\{{\xi }_i,i\ge 1\}$ is a stationary m-dependent sequence, so is $\{{[{\phi }^{\prime }(0)]}^{-1}\psi (e_i)x_{in},i\ge 1\}$. Let $u\in R^p$, $|u|=1$. Then $E(u^T{[{\phi }^{\prime }(0)]}^{-1}\psi (e_i)x_{in})=0$ and

$${\sigma }_i^2=E{(u^T{[{\phi }^{\prime }(0)]}^{-1}\psi (e_i)x_{in})}^2={[{\phi }^{\prime }(0)]}^{-2}{u}^T{x}_{in}{x}_{in}^{T}uE{[\psi (e_i)]}^2.$$

Therefore, by $r_n=o(1)$ and $0<{\sigma }_{\psi }^2=E{[\psi (e_i)]}^2<\mathrm{\infty }$, we have

$$\begin{array}{rl}{\tau }^2=& n^{-1}\sum _{i=1}^n{[{\phi }^{\prime }(0)]}^{-2}{u}^T{x}_{in}{x}_{in}^{T}uE{[\psi (e_i)]}^2\\ & +2n^{-1}\sum _{i=2}^{m+1}(n-i)Cov(u^T{[{\phi }^{\prime }(0)]}^{-1}\psi (e_1)x_{1n},u^T{[{\phi }^{\prime }(0)]}^{-1}\psi (e_i)x_{in})\\ =& {[{\phi }^{\prime }(0)]}^{-2}{n}^{-1}\{\sum _{i=1}^{n}E{[\psi (e_i)]}^2+2\sum _{i=2}^{m+1}(n-i)u^T{x}_{in}{x}_{in}^{T}uCov(\psi (e_1),\psi (e_i))\}\\ \to & {[{\phi }^{\prime }(0)]}^{-2}{\sigma }_{\psi }^2.\end{array}$$ 3.4

Thus the corollary follows from Lemma 3.2, (3.3) and (3.4). □

Corollary 3.2

Assume that (A1)–(A5) hold. If $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ and $m(t)=O(\sqrt{t})$ as $t\to 0$, and ${\tilde{\mathrm{\Omega }}}_n({\tilde{\beta }}_n)=O_{a.s.}({\tilde{r}}_n),0<{\sigma }_{\psi }^2=E{[\psi (e_i)]}^2<\mathrm{\infty }$, then

$${\tilde{\beta }}_n=O_{a.s.}\left(n^{-1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }\right).$$

Proof

The corollary follows from Proposition 3.1 and Corollary 2.2. □

$(\epsilon ,\psi )$-weakly dependent process

In the subsection, we assume that $\{{\epsilon }_i\}$ are $(\epsilon ,\psi )$-weakly dependent (Doukhan and Louhichi [14] and Dedecker et al. [11]) random variables. In 1999, Doukhan and Louhichi proposed a new idea of $(\epsilon ,\psi )$-weakly dependence which focuses on covariance rather than the total variation distance between joint distributions and the product of the corresponding marginal. It has been shown that this concept is more general than mixing and includes, under natural conditions on the process parameters, essentially all classes of processes of interest in statistics. Therefore, many researchers are interested in the $(\epsilon ,\psi )$-weakly dependent and related possesses, and one obtained lots of sharp results. For example, Doukhan and Louhichi [14], Dedecker and Doukhan [10], Dedecker and Prieur [12], Doukhan and Neumann [16], Doukhan and Wintenberger [17], Bardet et al. [3], Doukhan and Wintenberger [18], Doukhan et al. [13]. However, a few people (only Hwang and Shin [29], Nze et al. [40]) investigated regression models with $(\epsilon ,\psi )$-weakly dependent errors. Nobody has investigated a robust estimate for the regression model with $(\epsilon ,\psi )$-weakly dependent errors. To give the definition of the $(\epsilon ,\psi )$-weakly dependence, let us consider a process $\xi =\{{\xi }_n,n\in Z\}$ with values in a Banach space $(\mathcal{E},\parallel \cdot \parallel )$. For $h:{\mathcal{E}}^u\to R$, $u\in N$, we define the Lipschitz modulus of h,

$$\mathrm{Lip}h=\underset{y\ne x}{sup}|h(y)-h(x)|/{\parallel y-x\parallel }_1,$$ 3.5

where we have the $l_1$-norm, i.e., ${\parallel (y_1,y_2,\dots ,y_u)\parallel }_1={\sum }_{i=1}^u|y_i|$.

Definition 1

(Doukhan and Louhich [14])

A process $\xi =\{{\xi }_n,n\in Z\}$ with values in $R^d$ is called a $(\epsilon ,\psi )$-weakly dependent process if, for some classes of functions ${\mathcal{E}}^u,{\mathcal{E}}^v\to R,F_u,G_v$:

$$\begin{array}{rl}\epsilon (r)& =\underset{u,v}{sup}\underset{s_1\ge s_2\ge \cdots \ge s_u,t_1\ge t_2\ge \cdots \ge t_v,r=t_1-s_u}{sup}\underset{f\in F_u,g\in G_v}{sup}\frac{|Cov(f({\xi }_{s_1},{\xi }_{s_2},\dots ,{\xi }_{s_u}),g({\xi }_{t_1},{\xi }_{t_2},\dots ,{\xi }_{t_v}))|}{\mathrm{\Psi }(f,g)}\\ & \to 0\end{array}$$

as $r\to \mathrm{\infty }$.

According to the definition, mixing sequences ($\alpha ,\rho ,\beta ,\phi$-mixing), associated sequences (positively or negatively associated), Gaussian sequences, Bernoulli shifts and Markovian models or time series bootstrap processes with discrete innovations are $(\epsilon ,\psi )$-weakly dependent (Doukhan et al. [15]).

From now on, assume that the classes of functions contain functions bounded by 1. Distinct functions Ψ yield $\eta ,\theta ,\kappa$ and a λ weak dependence of the coefficients as follows (Doukhan et al. [15]):

$$\mathrm{\Psi }(f,g)=\{\begin{array}{cc}u\mathrm{Lip}f+v\mathrm{Lip}g\hfill & \text{then denote }\epsilon (r)=\eta (r),\hfill \\ v\mathrm{Lip}g\hfill & \text{then denote }\epsilon (r)=\theta (r),\hfill \\ uv\mathrm{Lip}f\cdot \mathrm{Lip}g\hfill & \text{then denote }\epsilon (r)=\kappa (r),\hfill \\ u\mathrm{Lip}f+v\mathrm{Lip}g+uv\mathrm{Lip}f\cdot \mathrm{Lip}g\hfill & \text{then denote }\epsilon (r)=\lambda (r),\hfill \\ u\mathrm{Lip}f+v\mathrm{Lip}g+uv\mathrm{Lip}f\cdot \mathrm{Lip}g+u+v\hfill & \text{then denote }\epsilon (r)=\omega (r).\hfill \end{array}$$ 3.6

In Corollary 3.3, we only consider λ and η-weakly dependence. Let $\{{\epsilon }_i\}$ be λ or η-weakly dependent, and assume that g satisfies: for each $s\in Z$, if $x,y\in R^Z$ satisfy $x_i=y_i$ for each index $i\ne s$

$$|g(x)-g(y)|\le b_s(\underset{i\ne s}{sup}|x_i{|}^l\vee 1)|x_s-y_s|.$$ 3.7

Lemma 3.3

(Dedecker et al. [11])

Assume that g satisfies the condition (3.7) with $l\ge 0$ and some sequence $b_s\ge 0$ such that ${\sum }_s|s|b_s<\mathrm{\infty }$. Assume that $E{|{\epsilon }_0|}^{m^{\prime }}<\mathrm{\infty }$ with $lm<m^{\prime }$ for some $m>2$. Then:

1. If the process $\{{\epsilon }_i,i\in Z\}$ is λ-weakly dependent with coefficients ${\lambda }_{\epsilon }(r)$, then $e_n$ is λ-weakly dependent with coefficients

   $${\lambda }_e(k)=c\underset{r\le [k/2]}{inf}\left(\sum _{i\ge r}{b}_i\right)\vee \left[{(2r+1)}^2{\lambda }_{\epsilon }{(k-2r)}^{\frac{m^{\prime }-1-l}{m^{\prime }-1+l}}\right].$$ 3.8
2. If the process $\{{\epsilon }_i,i\in Z\}$ is η-weakly dependent with coefficients ${\eta }_{\epsilon }(r)$, then $e_n$ is η-weakly dependent and there exists a constant $c>0$ such that

   $${\eta }_e(k)=c\underset{r\le [k/2]}{inf}\left(\sum _{i\ge r}{b}_i\right)\vee \left[{(2r+1)}^{1+\frac{1}{m^{\prime }-1}}{\eta }_{\epsilon }{(k-2r)}^{\frac{m^{\prime }-2}{m^{\prime }-1}}\right].$$

Lemma 3.4

(Bardet et al. [3])

Let $\{{\xi }_n,n\in Z\}$ be a sequence of $R^k$-valued random variables. Assume that there exists some constant $C>0$ such that ${max}_{1\le i\le k}{\parallel {\xi }_i\parallel }_p\le C,p\ge 1$. Let h be a function from $R^k$ to R such that $h(0)=0$ and for $x,y\in R^k$, there exist a in $[1,p]$ and $c>0$ such that

$$|h(x)-h(y)|\le c|x-y|(1+|x{|}^{\alpha -1}+|y{|}^{\alpha -1}).$$ 3.9

Now we define the sequence $\{{\zeta }_n,n\in Z\}$ by ${\zeta }_n=h({\xi }_n)$. Then:

(1) If the process $\{{\xi }_i,i\in Z\}$ is λ-weakly dependent with coefficients ${\lambda }_{\xi }(r)$, then $\{{\zeta }_n,n\in Z\}$ is also with coefficients

$${\lambda }_{\zeta }(r)=O({\lambda }_{\xi }^{\frac{p-a}{p+a-2}}(r)).$$ 3.10

(2) If the process $\{{\xi }_i,i\in Z\}$ is ζ-weakly dependent with coefficients ${\eta }_{\xi }(r)$, so is $\{{\zeta }_n,n\in Z\}$ with coefficients ${\eta }_{\zeta }(r)=O({\eta }_{\xi }^{\frac{p-a}{p-1}}(r))$.

Lemma 3.5

(Dedecker et al. [11])

Let $\{{\xi }_i,i\in Z\}$ be a centered and stationary real-valued sequence with $E{|{\xi }_0|}^{2+\varsigma }<\mathrm{\infty }$, $\varsigma >0$, ${\sigma }^2={\sum }_{k\in Z}Cov({\xi }_0,{\xi }_k)$ and $S_n={\sum }_{i=1}^n{\xi }_i$. If ${\lambda }_{\xi }(r)=O(r^{-\lambda })$ for $\lambda >4+2/\varsigma$, then $n^{-1/2}{S}_n\to N(0,{\sigma }^2)$ as $n\to \mathrm{\infty }$.

Corollary 3.3

Let $\{{\epsilon }_i\}$ be λ-weakly dependent with coefficients ${\lambda }_{\epsilon }(r)=O(exp(-r\lambda ))$ for some $\lambda >0$, and $b_i=O(exp(-ib))$ for some $b>0$. Assume that $\psi (0)=0$, and, for $x,y\in R$, there exists a constant $c>0$ such that

$$|\psi (x)-\psi (y)|\le c|x-y|.$$ 3.11

Under the conditions of Corollary 2.1, we have

$${\phi }^{\prime }(0)n^{-1/2}{T}_n\to N(0,\mathrm{\Sigma })\mathit{\text{as }}n\to \mathrm{\infty },$$ 3.12

where $\mathrm{\Sigma }={\sum }_{i=1}^n{x}_{1n}Cov(\psi (e_1),\psi (e_i))x_{in}^T$.

Proof

Note that $\{{\epsilon }_i\}$ is λ-weakly dependent. By Lemma 3.3, we find that $\{e_i\}$ is λ-weakly dependent with coefficients

$${\lambda }_e(r)=O(r^{2}exp(-\lambda r\frac{b(m^{\prime }-1-l)}{b(m^{\prime }-1+l+2\alpha (m^{\prime }-1-l))})),\alpha >0$$ 3.13

from (3.8) and Proposition 3.1 in Chap. 3 (Dedecker et al. [11]).

Let $u\in R^p$, $|u|=1$, and ${\zeta }_i=h(e_i)=u\psi (0)x_{in}=0$. Then $h(0)=0=u\psi (0)x_{in}=0$. Choose $p=2,a=1$, in (3.9), and by (3.11), we have

$$|h(x)-h(y)|=|x_{in}||\psi (x)-\psi (y)|\le c|x-y|$$ 3.14

for $x,y\in R$ and $c>0$. Therefore, by Lemma 3.4, $\{{\zeta }_i,i\in N\}$ is λ-weakly dependent with coefficients

$${\lambda }_{\zeta }(r)=O(r_n^{uv}{\lambda }_e^{\frac{p-a}{p+a-2}}(r))=O(r_n^{uv}{\lambda }_e(r)).$$ 3.15

By Corollary 2.1, we have

$${\phi }^{\prime }(0)n^{-1/2}{\hat{\beta }}_n=n^{-1/2}\sum _{i=1}^n\psi (e_i)x_{in}+o_p(1).$$ 3.16

By (3.13) and (3.15), there exist $b>0,a>0,l\ge 0$ and $m^{\prime }>lm$ for some $m>2$ such that

$${\lambda }_{\zeta }(r)=O(r_n^{uv}{r}^{2}exp(-\lambda r\frac{b(m^{\prime }-1-l)}{b(m^{\prime }-1+l)+2\alpha (m^{\prime }-1-l)}))=O\left(r^{-\lambda }\right)$$ 3.17

for enough large r and $\lambda >4+2/\varsigma$ with $\varsigma >0$.

By Lemma 3.5 and (3.16)–(3.17), we have

$${\phi }^{\prime }(0)n^{-1/2}u{T}_n\to N(0,{\sigma }^2),$$

where ${\sigma }^2={\sum }_{i=1}^n{u}^T{x}_{1n}Cov(\psi (e_1),\psi (e_i))x_{in}^{T}u$. Using the Cramer device, we complete the proof of Corollary 3.3. □

Lemma 3.6

(Dedecker et al. [11])

Suppose that $\{{\xi }_i,1\le i\le n\}$ are stationary real-valued random variables with $E{\xi }_i=0$ and $P(|{\xi }_i|\le M<\mathrm{\infty })=1$ for all $i=1,2,\dots ,n$. Let $\mathrm{\Psi }:N^2\to N$ be one of the following functions:

$$\begin{array}{rl}& \mathrm{\Psi }(u,v)=2v,\mathrm{\Psi }(u,v)=u+v,\mathrm{\Psi }(u,v)=uv,\\ & \mathrm{\Psi }(u,v)=\alpha (u+v)+(1-\alpha )uv,\end{array}$$ 3.18

for some $0<\alpha <1$. We assume that there exist constants $K,L_1,L_2<\mathrm{\infty },\mu \ge 0$ and a nonincreasing sequence of real coefficients $\{\rho (n),n\ge 0\}$ such that, for all u-tuples $(s_1,\dots ,s_u)$ and all v-tuples $(t_1,\dots ,t_v)$ with $1\le s_1\le \cdots \le s_u\le t_1\le \cdots \le t_v\le n,$ the following inequality is fulfilled:

$$|Cov({\xi }_{s_1},\dots ,{\xi }_{s_u};{\xi }_{t_1},\dots ,{\xi }_{t_v})|\le K^2{M}^{u+v-2}\mathrm{\Psi }(u,v)\rho (t_1-s_u),$$ 3.19

where

$$\sum _{s=0}^{\mathrm{\infty }}{(s+1)}^k\rho (s)\le L_1{L}_2^k{(k!)}^{\mu },\mathrm{\forall }k\ge 0.$$ 3.20

Let $S_n={\sum }_{i=1}^n{\xi }_i$ and ${\sigma }_n^2=Var({\sum }_{i=1}^n{\xi }_i)$. If ${\sigma }^2={lim}_{n\to \mathrm{\infty }}{\sigma }_n^2/n>0$, then

$$\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}sup}\frac{|S_n|}{\sigma {(2nloglogn)}^{1/2}}\le 1.$$ 3.21

Corollary 3.4

Let $\{{\epsilon }_i\}$ be η-weakly dependent with coefficients ${\eta }_{\epsilon }(r)=O(exp(-r\eta ))$ for some $\eta >0$, and $b_i=O(exp(-ib))$ for some $b>0$. Assume that $\psi (0)=0$ and (3.11) hold. Under the conditions of Corollary 2.2 with ${\tilde{r}}_n=O(n^{1/2}{(logn)}^{-2})$ replaced by $0<{min}_{1\le i\le n}|x_{ij}|<{max}_{1\le i\le n}|x_{ij}|<\mathrm{\infty }$, and $0<{\sigma }_{\psi }^2=E{\psi }^2(e_i)<\mathrm{\infty }$, we have:

1. for $3/2<q\le 7/4,{\mathrm{\Sigma }}_n{\tilde{\beta }}_n=O_{a.s.}(n{b}_n)=O_{a.s.}(n^{1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon })$;

2. for $q\ge 7/4,{\mathrm{\Sigma }}_n{\tilde{\beta }}_n=O_{a.s.}(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+2/\upsilon })$.

Proof

Let ${\xi }_i=\psi (e_i)x_{ij},j=1,\dots ,p$. Then for $\mathrm{\forall }{\mu }_n\to \mathrm{\infty }$ as $n\to \mathrm{\infty }$

$$P\left\{\right|\psi (e_i)x_{ij}|>{\mu }_n\}\le \frac{E|\psi (e_i)x_{ij}{|}^2}{{\mu }_n^2}=\frac{{\sigma }_{\psi }^2{max}_{1\le i\le n}|x_{ij}^2|}{{\mu }_n^2}\to 0.$$ 3.22

Therefore, there exists some $0<M<\mathrm{\infty }$ such that

$$P\left\{\right|\psi (e_i)x_{ij}|\le M\}=1.$$ 3.23

Similar to the proofs of (3.13) and (3.15), we easily obtain

$${\eta }_{\zeta }(r)=O({\tilde{r}}_n^{uv}{\eta }_e^{\frac{p-a}{p+a-2}}(r))=O({\tilde{r}}_n^{uv}{\eta }_e(r)),$$ 3.24

where

$${\eta }_e(r)=O(r^{\frac{m^{\prime }-1-l}{m^{\prime }-1}}exp(-\eta r\frac{b(m^{\prime }-2)}{b(m^{\prime }-1)+2\eta (m^{\prime }-2)})).$$ 3.25

By (3.24) and (3.25), we have

$$\begin{array}{rl}& |Cov({\xi }_{s_1},\dots ,{\xi }_{s_u};{\xi }_{t_1},\dots ,{\xi }_{t_v})|\\ & \le (u+v){\eta }_{\zeta }(r)\le (u+v){\tilde{r}}_n^{uv}{\eta }_e(r)\\ & \le (u+v){\tilde{r}}_n^{uv}{r}^{\frac{m^{\prime }-1-l}{m^{\prime }-1}}exp(-\eta r\frac{b(m^{\prime }-2)}{b(m^{\prime }-1)+2\eta (m^{\prime }-2)}).\end{array}$$ 3.26

Let $\mathrm{\Psi }(u,v)=u+v,K^2=r_n^{uv}{M}^{1(u+v-2)}$ and

$$\rho (s)=r^{\frac{m^{\prime }-1-l}{m^{\prime }-1}}exp(-\eta r\frac{b(m^{\prime }-2)}{b(m^{\prime }-1)+2\eta (m^{\prime }-2)}).$$ 3.27

Thus (3.19) holds. Since ${lim}_{s\to \mathrm{\infty }}ln(s+1)/s=0$, there exist $b>0,\eta >0,l\ge 0$ and $m^{\prime }>lm$ for some $m>2$ and $m^{\prime }>2$ such that

$$exp(-\eta s\frac{b(m^{\prime }-2)}{b(m^{\prime }-1)+2\eta (m^{\prime }2)})\le {(s+1)}^{-(2+k)},\mathrm{\forall }k\ge 0.$$ 3.28

Thus

$$\begin{array}{rl}& \sum _{s=0}^{\mathrm{\infty }}{(s+1)}^k\rho (s)\le \sum _{s=0}^{\mathrm{\infty }}{(s+1)}^{k+\frac{m^{\prime }-1-l}{m^{\prime }-1}}exp(-\eta s\frac{b(m^{\prime }-2)}{b(m^{\prime }-1)+2\eta (m^{\prime }-2)})\\ & \phantom{\sum _{s=0}^{\mathrm{\infty }}{(s+1)}^k\rho (s)}\le \sum _{s=0}^{\mathrm{\infty }}{(s+1)}^{-2+\frac{m^{\prime }-1-l}{m^{\prime }-1}}<\mathrm{\infty },\end{array}$$ 3.29

$$\begin{array}{rl}& {\sigma }^2=\underset{n\to \mathrm{\infty }}{lim}\{n^{-1}\sum _{i=1}^{n}E{\psi }^2(e_i)x_{ij}^2+n^{-1}\sum _{i,k=1;i\ne k}^n{x}_{ij}{x}_{kj}Cov(\psi (e_i),\psi (e_k))\}\\ & \phantom{{\sigma }^2}=\underset{n\to \mathrm{\infty }}{lim}\{n^{-1}\sum _{i=1}^{n}E{\psi }^2(e_i)x_{ij}^2+\frac{n-1}{2}O\left(x_{ij}^2\right)\sum _{i=1}^{n-1}(n-i)Cov(\psi (e_1),\psi (e_{i+1}))\}\\ & \phantom{{\sigma }^2}={\sigma }_{\psi }^2{\overline{x}}_{\cdot j}>0.\end{array}$$ 3.30

By Lemma 3.6 and Corollary 2.3, we have

$$\begin{array}{rl}{\mathrm{\Sigma }}_n{\tilde{\beta }}_n& =O_{a.s.}\left({(2nloglogn)}^{1/2}\right)+O_{a.s.}\left(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+\upsilon /2}\right)\\ & =O_{a.s.}\left(n^{1/2}{(logn)}^{-1/4+q}{(loglogn)}^{1/4+\upsilon /2}\right).\end{array}$$ 3.31

Therefore, by Corollary 2.3, (3.23) and (3.31), we complete the proof of Corollary 3.4. □

Linear martingale difference processes

In the subsection, we will investigate martingale difference errors $\{{\epsilon }_i\}$. We shall provide some sufficient conditions for (A6) and (A7) and give the central limit theorem and strong convergence rates.

Let $\{{\epsilon }_i\}$ be a martingale difference sequence, and $a_j$ be real numbers such that $e_i={\sum }_{j=0}^{\mathrm{\infty }}{a}_j{\epsilon }_{i-j}$ exists. It is well known that the theory of martingales provides a natural unified method for dealing with limit theorems. Under its influence, there is great interest in the martingale difference. Liang and Jing [34] were concerned with the partial linear model under the linear com of martingale differences and obtained asymptotic normality of the least squares estimator of the parameter. Nelson [39] has given conditions for the pointwise consistency of weighted least squares estimators from multivariate regression models with martingale difference errors. Lai [31] investigated stochastic regression models with martingale difference sequence errors and obtained strong consistency and asymptotic normality of the least squares estimate of the parameter.

Let $F_{\epsilon }$ be the distribution function of ${\epsilon }_0$ and let $f_{\epsilon }$ be its density.

Proposition 3.2

Suppose that $E{\epsilon }_0=0,{\epsilon }_0\in L^{4/(2-\gamma )}$, ${\kappa }_{\gamma }={\int }_R{\psi }^2(u){\omega }_{-\gamma }(du)<\mathrm{\infty },1<\gamma <2$ and ${\sum }_{k=0}^p{\int }_R{|f_{\epsilon }^{(k)}(v)|}^2{\omega }_{\gamma }(dv)<\mathrm{\infty }$, where ${\omega }_{\gamma }(dv)={(1+|v|)}^{\gamma }$. If ${\sum }_{j=0}^{\mathrm{\infty }}|a_j|<\mathrm{\infty }$, then ${\sum }_{i=0}^{\mathrm{\infty }}\omega (i)<\mathrm{\infty },{\sum }_{i=0}^{\mathrm{\infty }}\overline{\omega }(i)<\mathrm{\infty }$ and ${\sum }_{i=0}^{\mathrm{\infty }}\tilde{\omega }(i)<\mathrm{\infty }$.

Proof

Let $Z_n={\sum }_{j=0}^{\mathrm{\infty }}{a}_j{\epsilon }_{n-j},Z_n^{\ast }=Z_n-a_n{\epsilon }_0-a_n{\epsilon }^{\prime }$, and

$$R_n={\int }_R{[f_{\epsilon }(t-U_n)-f_{\epsilon }(t-U_n-a_0{\epsilon }_n)]}^2{\omega }_{\gamma }(dt),$$ 3.32

where $U_n=Z_n-a_n{\epsilon }_0$. By the Schwartz inequality, we have

$$\begin{array}{rl}{\omega }^2(n)& ={({\int }_R\parallel f_{\epsilon }(t-Z_n)-f_{\epsilon }(t-Z_n^{\ast })\parallel {(1+|t|)}^{\gamma }\cdot \psi (t;{\epsilon }_0){(1+|t|)}^{-\gamma }dt)}^2\\ & \le {\int }_R{\psi }^2(t;{\epsilon }_0){\omega }_{-\gamma }(dt)\cdot {\int }_R{\parallel f_{\epsilon }(t-Z_n)-f_{\epsilon }(t-Z_n^{\ast })\parallel }^2{\omega }_{\gamma }(dt)\\ & ={\kappa }_{\gamma }{\int }_R{\parallel f_{\epsilon }(t-Z_n)-f_{\epsilon }(t-Z_n^{\ast })\parallel }^2{\omega }_{\gamma }(dt)\le CE(R_n).\end{array}$$ 3.33

Note that

$$f_{\epsilon }(t-U_n)-f_{\epsilon }(t-U_n-a_n{\epsilon }_0)={\int }_0^{a_n{\epsilon }_0}{f}_{\epsilon }^{\prime }(t-U_n-v)dv$$ 3.34

and

$$\begin{array}{rl}{\int }_R{[f_{\epsilon }^{\prime }(t-u)]}^2{\omega }_{\gamma }(dt)& ={(1+|u|)}^{\gamma }{\int }_R{[f_{\epsilon }^{\prime }(v)]}^2{(1+|u|)}^{-\gamma }{(1+|u+v|)}^{\gamma }dv\\ & \le {(1+|u|)}^{\gamma }{\int }_R{[f_{\epsilon }^{\prime }(v)]}^2{\omega }_{\gamma }(dv)\le I_1{(1+|u|)}^{\gamma }.\end{array}$$ 3.35

Let $I_k={\int }_R{[f_{\epsilon }^{(k)}(v)]}^2{\omega }_{\gamma }(dv)$. By the Schwartz inequality, we have

$$\begin{array}{rl}{R}_n& \le {\int }_R|{\int }_0^{a_n{\epsilon }_0}{1}^{2}dv\cdot {\int }_0^{a_n{\epsilon }_0}{[f_{\epsilon }^{\prime }(t-U_n-v)]}^{2}dv|{\omega }_{\gamma }(dt)\\ & \le |a_n{\epsilon }_0|{\int }_0^{a_n{\epsilon }_0}{I}_1{(1+|U_n+v|)}^{\gamma }dv\\ & \le |a_n{\epsilon }_0{|}^2[{(1+|U_n|)}^{\gamma }+{(1+|U_n+a_n{\epsilon }_0|)}^{\gamma }]\\ & \le C|a_n{\epsilon }_0{|}^2[{(1+|U_n|)}^{\gamma }+|a_n{\epsilon }_0{|}^{\gamma }].\end{array}$$ 3.36

By ${sup}_{j}E{\epsilon }_j^2<\mathrm{\infty }$ and Chatterji’s inequality (Lin and Bai [35]), we have

$$E{U}_n^2\le \sum _{j\ne n,j=1}^{\mathrm{\infty }}{a}_j^{2}E{\epsilon }_{n-j}^2\le \sum _{j=0}^{\mathrm{\infty }}{a}_j^2.$$ 3.37

By (3.33)–(3.37) and the Schwartz inequality, we have

$$\begin{array}{rl}E(R_n)& \le CE\{|a_n{\epsilon }_0{|}^2+|a_n{\epsilon }_0{|}^{2+\gamma }+|a_n{\epsilon }_0{|}^2|U_n{|}^{\gamma }\}\\ & \le C\{|1+|a_n{|}^{\gamma }+E[|{\epsilon }_0{|}^2|U_n{|}^{\gamma }]\}\\ & \le C{a}_n^2\{|1+|a_n{|}^{\gamma }+{(E|U_n{|}^2)}^{\gamma /2}\}\\ & \le C{a}_n^2\{|1+|a_n{|}^{\gamma }+{\left(\sum _{j=0}^{\mathrm{\infty }}{a}_j^2\right)}^{\gamma /2}\}.\end{array}$$ 3.38

Note that ${\sum }_{j=0}^{\mathrm{\infty }}|a_j|<\mathrm{\infty }$ implies ${\sum }_{j=0}^{\mathrm{\infty }}{a}_j^2<\mathrm{\infty }$ and ${\sum }_{j=0}^{\mathrm{\infty }}{|a_j|}^{1+\gamma /2}<\mathrm{\infty }$, and by (3.33) and (3.39), we have

$$\sum _{i=0}^{\mathrm{\infty }}\omega (i)\le \sum _{n=0}^{\mathrm{\infty }}max(|a_n|,|a_n{|}^{1+\gamma /2})<\mathrm{\infty }.$$ 3.39

The general case $k\ge 1$ similarly follows. Similar to the proof of (3.39), we easily prove the other results. □

From Propositions 2.1 and 3.2, (A6) and (A7) hold. Hence, we can obtain the following two corollaries from Corollaries 2.1 and 2.2. In order to prove the following two corollaries, we first give some lemmas.

Lemma 3.7

(Liptser and Shiryayev [36])

Let $\xi ={({\xi }_k)}_{-\mathrm{\infty }<k<\mathrm{\infty }}$ be a strictly stationary sequence on a probability space $(\mathrm{\Omega },\mathcal{F},P)$, and $\mathcal{G}$ be a σ-algebra of invariant sets of the sequence ξ and ${\mathcal{F}}_k=\sigma (\dots ,{\xi }_{k-1},{\xi }_k)$. For a certain $p\ge 2$, let $E{|{\xi }_0|}^p<\mathrm{\infty }$ and ${\sum }_{k\ge 1}{\gamma }_k(p)<\mathrm{\infty }$, where ${\gamma }_k(p)={\{E|E({\xi }_k{|F_0)|}^{\frac{p}{p-1}}\}}^{\frac{p-1}{p}}$. Then

$$Z_n=\frac{1}{\sqrt{n}}\sum _{k=1}^n{\xi }_k\stackrel{d}{⟶}Z(\mathit{\text{stably}}),$$

where the random variable Z has the characteristic function $Eexp(-\frac{1}{2}{\lambda }^2{\sigma }^2)$, and ${\sigma }^2=E({\xi }_0^2|\mathcal{G})+2{\sum }_{k\ge 1}E({\xi }_0{\xi }_k|\mathcal{G})$.

Corollary 3.5

Assume that (A1)–(A5) hold, $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ and $m(t)=O({|t|}^{\lambda })$ for some $\lambda >0$ as $t\to 0$, $\mathrm{\Omega }({\hat{\beta }}_n)=O_p(r_n)$. Under the conditions of Proposition 3.2, $E{|\psi (e_k)|}^{\frac{p}{p-1}}<\mathrm{\infty },p\ge 2$ and ${\sum }_{k=1}^n|x_{kn}|<\mathrm{\infty }$, we have

$$n^{-1/2}{\hat{\beta }}_n\stackrel{d}{⟶}Z(\mathit{\text{stably}}),$$ 3.40

where the random variable Z has the characteristic function $Eexp(-\frac{1}{2}{\lambda }^2{\sigma }^2)$, and ${\sigma }^2={({\phi }^{\prime }(0))}^{-2}{x}_{1n}^T{x}_{1n}E({\psi }^2(e_1)|\mathcal{G})+2{({\phi }^{\prime }(0))}^{-2}{x}_{1n}^T{\sum }_{k\ge 2}{x}_{kn}E(\psi (e_1)\psi (e_k)|\mathcal{G})$.

Proof

By Proposition 2.1, Proposition 3.2 and Corollary 2.1, we have

$$\begin{array}{rl}{n}^{-1/2}\widehat{{\beta }_n}& =n^{-1/2}{({\phi }^{\prime }(0))}^{-1}\sum _{i=1}^n\psi (e_i)x_{in}+O_p\left(n^{-1/2}(r_n^{\lambda }{log}^{1/2}n+r_n)\right)\\ & =n^{-1/2}{({\phi }^{\prime }(0))}^{-1}\sum _{i=1}^n\psi (e_i)x_{in}+o_p(1).\end{array}$$ 3.41

By $E{|\psi (e_k)|}^{\frac{p}{p-1}}<\mathrm{\infty }$ and ${\sum }_{k=1}^n|x_{kn}|<\mathrm{\infty }$, we have

$$\begin{array}{rl}& {\gamma }_k(p)={\left\{E\right|E(\psi (e_k)x_{kn}|{\mathcal{F}}_0){|}^{\frac{p}{p-1}}\}}^{\frac{p-1}{p}}\\ & \phantom{{\gamma }_k(p)}\le {\left\{E\left[E\left(\right|\psi (e_k)x_{kn}{|}^{\frac{p}{p-1}}|{\mathcal{F}}_0)\right]\right\}}^{\frac{p-1}{p}}\\ & \phantom{{\gamma }_k(p)}={\left\{E\right|\psi (e_k)x_{kn}{|}^{\frac{p}{p-1}}\}}^{\frac{p-1}{p}}\}\le C|x_{kn}|,\end{array}$$ 3.42

$$\sum _{k\ge 1}{\gamma }_k(p)=\sum _{k=1}^n|x_{kn}|<\mathrm{\infty }$$ 3.43

and

$$\begin{array}{rl}{\sigma }^2& ={({\phi }^{\prime }(0))}^{-2}E({(\psi (e_1)x_{1n})}^2|\mathcal{G})+2{({\phi }^{\prime }(0))}^{-2}\sum _{k\ge 2}E(\psi (e_1)x_{1n}\psi (e_k)x_{kn}|\mathcal{G})\\ & ={({\phi }^{\prime }(0))}^{-2}{x}_{1n}^{2}E({\psi }^2(e_1)|\mathcal{G})+2{({\phi }^{\prime }(0))}^{-2}{x}_{1n}\sum _{k\ge 2}{x}_{kn}E(\psi (e_1)\psi (e_k)|\mathcal{G}).\end{array}$$

By Proposition 2.1, Proposition 3.2 and Corollary 2.2, we easily obtain the following result. Here we omit the proof. □

Corollary 3.6

Assume that (A1)–(A5) hold, $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ and $m(t)=O(\sqrt{t})$ as $t\to 0$, ${\tilde{\mathrm{\Omega }}}_n({\tilde{\beta }}_n)=O_{a.s.}({\tilde{r}}_n)$. Under the conditions of Proposition 3.2, we have

$${\tilde{\beta }}_n=O_{a.s}\left(n^{-1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }\right),\upsilon >0.$$

Proofs of the main results

For the proofs of Theorem 2.1 and Theorem 2.2, we need some lemmas as follows.

Lemma 4.1

(Freedman [21])

Let τ be a stopping time, and K a positive real number. Suppose that $P\{|{\xi }_i|\le K,i\le \tau \}=1$, where $\{{\xi }_i\}$ are measurable random variables and $E({\xi }_i|{\mathcal{F}}_{i-1})=0$. Then, for all positive real numbers a and b,

$$\begin{array}{rl}P\{\sum _{i=1}^n{\xi }_i\ge a\mathit{\text{ and }}{T}_n\le b,\mathit{\text{ for some }}n\le \tau \}& \le {\left({\left(\frac{b}{Ka+b}\right)}^{Ka+b}{e}^{Ka}\right)}^{K^{-2}}\\ & \le exp(-\frac{a^2}{2(Ka+b)}).\end{array}$$

Lemma 4.2

Let

$$M_n({\beta }_n)=\sum _{i=1}^n\{\psi (e_i-x_{in}^T{\beta }_n)-E(\psi (e_i-x_{in}^T{\beta }_n)|{\mathcal{F}}_{i-1})\}{x}_{in}.$$ 4.1

Assume that (A5) and (A6) hold. Then

$$\underset{|{\beta }_n|\le {\delta }_n}{sup}|M_n({\beta }_n)-M_n(0)|=O_p(\sqrt{{\tau }_n({\delta }_n)}logn+n^{-3}).$$ 4.2

Proof

Note that $p={\sum }_{i=1}^n{x}_{in}^T{x}_{in}\le {({max}_{1\le i\le n}|x_{in}|)}^{2}n=n{r}_n^2$, and ${\delta }_n{r}_n\to 0$, we have ${\delta }_n=o(n^{1/2})$. For any positive sequence ${\mu }_n\to \mathrm{\infty }$, let

$$\begin{array}{rl}& {\varphi }_n=2{\mu }_n\sqrt{{\tau }_n({\delta }_n)}logn,t_n={\mu }_n\sqrt{{\tau }_n({\delta }_n)}/log{\mu }_n,u_n=t_n^2,\\ & {\eta }_i({\beta }_n)=(\psi (e_i-x_{in}^T{\beta }_n)-\psi (e_i))x_{in},T_n=\underset{1\le i\le n}{max}\underset{|{\beta }_n|\le {\delta }_n}{sup}|{\eta }_i({\beta }_n)|\end{array}$$

and

$$U_n=\sum _{i=1}^{n}E\{{[\psi (e_i+|x_{in}|{\delta }_n)-\psi (e_i-|x_{in}|{\delta }_n)]}^2|{\mathcal{F}}_{i-1}\}|x_{in}{|}^2.$$

By the monotonicity of ψ and $\delta \ge 0$, we have

$$\begin{array}{rl}\underset{|{\beta }_n|\le \delta }{sup}|{\eta }_i({\beta }_n)|& \le |x_{in}|\underset{|{\beta }_n|\le \delta }{sup}|\psi (e_i-x_{in}^T{\beta }_n)-\psi (e_i)|\\ & \le |x_{in}|max\{\psi (e_i-|x_{in}|\delta )-\psi (e_i),\psi (e_i+|x_{in}|\delta )-\psi (e_i)\}\\ & \le |x_{in}|\{\psi (e_i+|x_{in}|\delta )-\psi (e_i-|x_{in}|\delta )\}.\end{array}$$ 4.3

By (4.3), the $c_r$-inequality and (A3), we have

$$\begin{array}{rl}E\left(\underset{|{\beta }_n|\le {\delta }_n}{sup}\right|{\eta }_i({\beta }_n){|}^2)& \le E{\{|x_{in}|[\psi (e_i+|x_{in}|\delta )-\psi (e_i-|x_{in}|\delta )]\}}^2\\ & \le 2|x_{in}{|}^2\{E{[\psi (e_i+|x_{in}|\delta )-\psi (e_i)]}^2+E{[\psi (e_i-|x_{in}|\delta )-\psi (e_i)]}^2\}\\ & =2|x_{in}{|}^2[m^2(|x_{in}|\delta )+m^2(-|x_{in}|\delta )].\end{array}$$

Thus

$$\begin{array}{rl}E\left(T_n^2\right)& =E\left(\underset{1\le i\le n}{max}\underset{|{\beta }_n|\le {\delta }_n}{sup}\right|{\eta }_i({\beta }_n){|}^2)\le \sum _{i=1}^{n}E\left(\underset{|{\beta }_n|\le {\delta }_n}{sup}\right|{\eta }_i({\beta }_n){|}^2)\\ & \le 2\sum _{i=1}^n|x_{in}{|}^2[m^2(|x_{in}|{\delta }_n)+m^2(-|x_{in}|{\delta }_n)]=2{\tau }_n({\delta }_n).\end{array}$$ 4.4

By the Chebyshev inequality,

$$P(|T_n|\ge t_n)\le E\left(T_n^2\right)/t_n^2\le 2{\tau }_n({\delta }_n)/t_n^2=2{log}^2{\mu }_n/{\mu }_n^2\to 0.$$ 4.5

Similarly,

$$P(|U_n|\ge t_n)\le E(U_n)/u_n=O\left({(log{\mu }_n/{\mu }_n)}^2\right)\to 0.$$ 4.6

Let $x_{in}={(x_{i1n},\dots ,x_{ipn})}^T={(x_{i1},\dots ,x_{ip})}^T,D_x(i)=(2\times 1_{x_{i1}\ge 0}-1,\dots ,2\times 1_{x_{ip}\ge 0}-1)\in {\mathrm{\Pi }}_p,{\mathrm{\Pi }}_p={\{-1,1\}}^p$. For $d\in {\mathrm{\Pi }}_p,j=1,2,\dots ,p$, define

$$M_{n,j,d}({\beta }_n)=\sum _{i=1}^n[\psi (e_i-x_{in}^T{\beta }_n)-E(\psi (e_i-x_{in}^T{\beta }_n)|F_{i-1})]x_{ij}{1}_{D_x(i)=d}.$$ 4.7

Since $M_n({\beta }_n)={\sum }_{d\in {\mathrm{\Pi }}_p}{(M_{n,1,d}({\beta }_n),\dots ,M_{n,p,d}({\beta }_n))}^T$, it suffices to prove that Lemma 4.2 holds with $M_n({\beta }_n)$ replaced by $(M_{n,j,d}({\beta }_n)$.

Let $|{\beta }_n|\le {\delta }_n,{\eta }_{i,j,d}({\beta }_n)=(\psi (e_i-x_{in}^T{\beta }_n)-\psi (e_i))x_{ij}{1}_{D_x(i)=d}$ and

$$B_n({\beta }_n)=\sum _{i=1}^{n}E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|>t_n}|{\mathcal{F}}_{i-1}).$$ 4.8

Note that

$$\frac{u_n}{t_n{\varphi }_n}=\frac{t_n}{{\varphi }_n}=\frac{{\mu }_n\sqrt{{\tau }_n({\delta }_n)}/log{\mu }_n}{2{\mu }_n\sqrt{{\tau }_n({\delta }_n)}logn}=\frac{1}{2lognlog{\mu }_n}\to 0.$$ 4.9

By (4.9), for large enough n, we have

$$\begin{array}{rl}P\left(\right|B_n({\beta }_n)|\ge {\varphi }_n,U_n\le u_n)& =P\left(\right|\sum _{i=1}^{n}E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|>t_n}|{\mathcal{F}}_{i-1})|\ge {\varphi }_n,U_n\le u_n)\\ & \le P(t_n^{-1}\sum _{i=1}^{n}E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|>t_n}|{\mathcal{F}}_{i-1})\ge {\varphi }_n,U_n\le u_n)\\ & \le P(t_n^{-1}{U}_n\ge {\varphi }_n,U_n\le u_n)=P(t_n{\varphi }_n\le U_n\le u_n)=0.\end{array}$$ 4.10

Let the projections ${\mathcal{P}}_k(\cdot )=E(\cdot |{\mathcal{F}}_k)-E(\cdot |{\mathcal{F}}_{k-1})$. Since

$$\begin{array}{rl}& E\{{\mathcal{P}}_i({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n})|{\mathcal{F}}_{i-1}\}\\ & =E\{[E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n}|{\mathcal{F}}_i)-E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n}|{\mathcal{F}}_{i-1})]|{\mathcal{F}}_{i-1}\}\\ & =E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n}|{\mathcal{F}}_{i-1})-E({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n}|{\mathcal{F}}_{i-1})=0.\end{array}$$ 4.11

Note that $\{{\mathcal{P}}_i({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n})\}$ are bound martingale differences. By Lemma 4.1 and (4.10), for $|{\beta }_n|\le t_n$, we have

$$\begin{array}{rl}& P\left\{\right|M_{n,j,d}({\beta }_n)-M_{n,j,d}(0)|\ge 2{\varphi }_n,T_n\le t_n,U_n\le u_n\}\\ & \le P\left\{\right|\sum _{i=1}^n{\mathcal{P}}_i({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|\le t_n})|\ge {\varphi }_n,T_n\le t_n,U_n\le u_n\}\\ & +P\left\{\right|\sum _{i=1}^n{\mathcal{P}}_i({\eta }_{i,j,d}({\beta }_n)1_{|{\eta }_{i,j,d}({\beta }_n)|>t_n})|\ge {\varphi }_n,T_n\le t_n,U_n\le u_n\}\\ & \le Cexp(-\frac{{\varphi }_n^2}{4t_n{\varphi }_n+2u_n})+P\left(\right|B_n({\beta }_n)|\ge {\varphi }_n,U_n\le u_n)\\ & =O(exp(-\frac{{\varphi }_n^2}{4t_n{\varphi }_n+2u_n})).\end{array}$$ 4.12

Let $l=n^8$ and $K_l=\{(k_1/l,\dots ,k_p/l):k_i\in Z,|k_i|\le n^9\}$. Then $\mathrm{\#}{K}_l={(2n^9+1)}^p$, where the symbol # denotes the number of elements of the set $K_l$. It is easy to show

$$t_n{\varphi }_{n}logn=o\left({\varphi }_n^2\right)\text{and}{u}_{n}logn=o\left({\varphi }_n^2\right).$$ 4.13

By (4.12) and (4.13), for $\mathrm{\forall }\varsigma >1$, we have

$$\begin{array}{rl}& P\left\{\underset{{\beta }_n\in K_l}{sup}\right|M_{n,j,d}({\beta }_n)-M_{n,j,d}(0)|\ge 2{\varphi }_n,T_n\le t_n,U_n\le u_n\}\\ & \le \sum _{\mathrm{\#}{K}_l}P\left\{\right|M_{n,j,d}({\beta }_n)-M_{n,j,d}(0)|\ge 2{\varphi }_n,T_n\le t_n,U_n\le u_n\}\\ & \le C{n}^{9p}exp(-\frac{{\varphi }_n^2}{4t_n{\varphi }_n+2u_n})=C{n}^{9p}exp(-\frac{logn}{4t_n{\varphi }_{n}logn/{\varphi }_n^2+2u_{n}logn/{\varphi }_n^2})\\ & =C{n}^{9p}exp(-\frac{logn}{o(1)})=o\left(n^{-\varsigma p}\right).\end{array}$$ 4.14

By (4.5), (4.6) and (4.14), we have

$$P\left\{\underset{{\beta }_n\in K_l}{sup}\right|M_{n,j,d}({\beta }_n)-M_{n,j,d}(0)|\ge 2{\varphi }_n\}\to 0,n\to \mathrm{\infty }.$$ 4.15

For a, let ${\langle a\rangle }_{l,-1}={\lceil a\rceil }_l=\lceil al\rceil /l$ and ${\langle a\rangle }_{l,1}={\lfloor a\rfloor }_l=\lfloor al\rfloor /l$. For a vector ${\beta }_n={({\beta }_{1n},\dots ,{\beta }_{pn})}^T$, let ${\langle {\beta }_n\rangle }_{l,d}=({\langle {\beta }_{1n}\rangle }_{l,d_1},\dots ,{\langle {\beta }_{pn}\rangle }_{l,d_p})$.

By (A5), for $|s|,|t|\le r_n{\delta }_n$ and large n, we have

$$\left|E\{[\psi (e_i-t)-\psi (e_i-s)]|{\mathcal{F}}_{i-1}\}\right|\le L_{i-1}|s-t|.$$

Let $V_n={\sum }_{i=1}^n{L}_{i-1}$. By condition (A5), the Markov inequality and $L_i\in L^1$, we have

$$P(V_n\ge n^4)\le E{V}_n/n^4=\sum _{i=1}^{n}E{L}_{i-1}/n^4\le C{n}^{-3}.$$ 4.16

Note that $|{\beta }_n-{\langle {\beta }_n\rangle }_{l,d}|\le C{l}^{-1}$, which implies ${max}_{1\le i\le n}|x_{in}^T({\beta }_n-{\langle {\beta }_n\rangle }_{l,d})|=o(l^{-1})$. Thus

$$\begin{array}{rl}& \underset{|{\beta }_n|\le {\delta }_n}{sup}\left|\sum _{i=1}^{n}E\{[{\eta }_i\left({\langle {\beta }_n\rangle }_{l,d}\right)-{\eta }_i({\beta }_n)]|{\mathcal{F}}_{i-1}\}{x}_{in}\right|\\ & \le \underset{|{\beta }_n|\le {\delta }_n}{sup}\sum _{i=1}^n\left|E((\psi (e_i-x_{in}^T{\langle {\beta }_n\rangle }_{l,d})-\psi (e_i))-(\psi (e_i-x_{in}^T{\beta }_n)-\psi (e_i))|{\mathcal{F}}_{i-1})x_{in}\right|\\ & =\underset{|{\beta }_n|\le {\delta }_n}{sup}\sum _{i=1}^n\left|E(\psi (e_i-x_{in}^T{\langle {\beta }_n\rangle }_{l,d})-\psi (e_i-x_{in}^T{\beta }_n)|{\mathcal{F}}_{i-1})x_{in}\right|\\ & \le \underset{|{\beta }_n|\le {\delta }_n}{sup}\sum _{i=1}^n|x_{in}|L_{i-1}\left|x_{in}^T({\langle {\beta }_n\rangle }_{l,d}-{\beta }_n)\right|\le C{l}^{-1}{V}_n.\end{array}$$ 4.17

Without loss of generality, assume that $j=1$ in the following proof.

Let $d=(1,-1,1,\dots ,1)$. Then ${\langle {\beta }_n\rangle }_{l,d}=({\lfloor {\beta }_{1n}\rfloor }_l,{\lceil {\beta }_{2n}\rceil }_l,{\lfloor {\beta }_{3n}\rfloor }_l,\dots ,{\lfloor {\beta }_{pn}\rfloor }_l)$ and ${\langle {\beta }_n\rangle }_{l,-d}=({\lceil {\beta }_{1n}\rceil }_l,{\lfloor {\beta }_{2n}\rfloor }_l,{\lceil {\beta }_{3n}\rceil }_l,\dots ,{\lceil {\beta }_{pn}\rceil }_l)$. Since ψ is nondecreasing,

$${\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)\le {\eta }_{i,1,d}({\beta }_n)\le {\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right).$$

Note that

$$\begin{array}{rl}& {\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)|{\mathcal{F}}_{i-1}]+E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)|{\mathcal{F}}_{i-1}]\\ & -E[{\eta }_{i,1,d}({\beta }_n)|{\mathcal{F}}_{i-1}]\\ & \le {\eta }_{i,1,d}({\beta }_n)-E[{\eta }_{i,1,d}({\beta }_n)|{\mathcal{F}}_{i-1}]\\ & \le {\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)|{\mathcal{F}}_{i-1}]+E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)|{\mathcal{F}}_{i-1}]\\ & -E[{\eta }_{i,1,d}({\beta }_n)|{\mathcal{F}}_{i-1}].\end{array}$$

Namely

$$\begin{array}{rl}& \sum _{i=1}^n\{{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)|{\mathcal{F}}_{i-1}]\\ & +E[({\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-{\eta }_{i,1,d}({\beta }_n))|{\mathcal{F}}_{i-1}]\}{x}_{i1}{1}_{D_x(i)=d}\\ & \le \sum _{i=1}^n\{{\eta }_{i,1,d}({\beta }_n)-E[{\eta }_{i,1,d}({\beta }_n)|{\mathcal{F}}_{i-1}]\}{x}_{i1}{1}_{D_x(i)=d}\\ & \le \sum _{i=1}^n\{{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-E[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)|{\mathcal{F}}_{i-1}]\\ & +E[({\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-{\eta }_{i,1,d}({\beta }_n))|{\mathcal{F}}_{i-1}]\}{x}_{i1}{1}_{D_x(i)=d}.\end{array}$$

Therefore

$$\begin{array}{rl}& M_{n,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-M_{n,1,d}(0)+\sum _{i=1}^{n}E\{[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-{\eta }_{i,1,d}({\beta }_n)]|{\mathcal{F}}_{i-1}\}{x}_{i1}{1}_{D_x(i)=d}\\ & \le M_{n,1,d}({\beta }_n)-M_{n,1,d}(0)\\ & \le M_{n,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-M_{n,1,d}(0)\\ & +\sum _{i=1}^{n}E\{[{\eta }_{i,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-{\eta }_{i,1,d}({\beta }_n)]|{\mathcal{F}}_{i-1}\}{x}_{i1}{1}_{D_x(i)=d}.\end{array}$$ 4.18

By (4.17) and (4.18), we have

$$\begin{array}{rl}{M}_{n,1,d}\left({\langle {\beta }_n\rangle }_{l,-d}\right)-M_{n,1,d}(0)-C{l}^{-1}{V}_n& \le M_{n,1,d}({\beta }_n)-M_{n,1,d}(0)\\ & \le M_{n,1,d}\left({\langle {\beta }_n\rangle }_{l,d}\right)-M_{n,1,d}(0)+C{l}^{-1}{V}_n.\end{array}$$ 4.19

Note that $l^{-1}{V}_n=O_p(n^{-8}{n}^4)=O_p(n^{-4})$, (4.2) immediately follows from (4.15) and (4.19). □

Lemma 4.3

Assume that the processes $X_t=g({\mathcal{F}}_t)\in L^2$. Let $g_n({\mathcal{F}}_0)=E(g({\mathcal{F}}_n)|{\mathcal{F}}_0),n\ge 0$. Then

$$\begin{array}{rl}& \parallel g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right)\parallel \le \parallel g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right)\parallel ,\\ & \parallel {\mathcal{P}}_0{X}_n\parallel \le \parallel g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right)\parallel +R,\end{array}$$ 4.20

where $R=\parallel E[g_n({\mathcal{F}}_0^{\ast })|{\mathcal{F}}_{-1}]-E[g_n({\mathcal{F}}_0^{\ast })|{\mathcal{F}}_0]\parallel$.

Proof

Since

$$\begin{array}{rl}E\{[g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right)]|({\mathcal{F}}_{-1},{\epsilon }_0^{\prime },{\epsilon }_0)\}& =E[g({\mathcal{F}}_n)|({\mathcal{F}}_{-1},{\epsilon }_0)]-E[g\left({\mathcal{F}}_n^{\ast }\right)|({\mathcal{F}}_{-1},{\epsilon }_0^{\prime })]\\ & =g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right),\end{array}$$

we have

$$E|E\{[g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right)]|({\mathcal{F}}_{-1},{\epsilon }_0^{\prime },{\epsilon }_0)\}{|}^2=E|g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right){|}^2.$$ 4.21

By the Jensen inequality, we have

$$\begin{array}{rl}E|E\{[g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right)]|({\mathcal{F}}_{-1},{\epsilon }_0^{\prime },{\epsilon }_0)\}{|}^2& \le E\left\{E\left[\right|g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right){|}^2|({\mathcal{F}}_{-1},{\epsilon }_0^{\prime },{\epsilon }_0)]\right\}\\ & =E|g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right){|}^2.\end{array}$$ 4.22

By (4.21) and (4.22), we have

$$E|g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right){|}^2\le E|g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right){|}^2.$$

That is,

$$\parallel g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right)\parallel \le \parallel g({\mathcal{F}}_n)-g\left({\mathcal{F}}_n^{\ast }\right)\parallel .$$ 4.23

Note that

$$E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_{-1}]=E[E(g({\mathcal{F}}_n)|{\mathcal{F}}_0)|{\mathcal{F}}_{-1}]=E(g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_{-1})$$ 4.24

and

$$E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_{-1}]=E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0]+E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_{-1}]-E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0].$$ 4.25

By (4.24), (4.25) and the Jensen inequality, we have

$$\begin{array}{rl}\parallel {\mathcal{P}}_0{X}_n\parallel & =\parallel E(g({\mathcal{F}}_n)|{\mathcal{F}}_0)-E(g({\mathcal{F}}_n)|{\mathcal{F}}_{-1})\parallel \\ & =\parallel E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_0]-E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_{-1}]\parallel \\ & =\parallel E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_0]-E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0]-E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_{-1}]+E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0]\parallel \\ & \le \parallel E[g_n({\mathcal{F}}_0)|{\mathcal{F}}_0]-E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0]\parallel +\parallel E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_{-1}]-E[g_n\left({\mathcal{F}}_0^{\ast }\right)|{\mathcal{F}}_0]\parallel \\ & \le \parallel g_n({\mathcal{F}}_0)-g_n\left({\mathcal{F}}_0^{\ast }\right)\parallel +R.\end{array}$$ 4.26

 □

Remark 4

If $\{{\epsilon }_i\}$ i.i.d., then $R=0$. In this case, the above lemma becomes Theorem 1 of Wu [48].

Lemma 4.4

Let $\{{\delta }_n,n\in N\}$ be a sequence of positive numbers such that ${\delta }_n\to \mathrm{\infty }$ and ${\delta }_n{r}_n\to 0$. If (A6)–(A7) hold, then

$$\parallel \underset{|{\beta }_n|\le {\delta }_n}{sup}|N_n({\beta }_n)-N_n(0)|\parallel =O\left(\sqrt{\sum _{i=1}^n|x_{in}{|}^4}{\delta }_n\right),$$ 4.27

where

$$N_n({\beta }_n)=\sum _{i=1}^n\{{\psi }_i(-x_{in}^T{\beta }_n;{\mathcal{F}}_{i-1})-\phi (-x_{in}^T{\beta }_n)\}{x}_{in}.$$

Proof

Let $I=\{n_1,\dots ,n_q\}\in \{1,2,\dots ,p\}$ be a nonempty set and $1\le n_1<\cdots <n_q$, and $u_I=(u_1{1}_{1\in I},\dots ,u_p{1}_{p\in I})$, with vector $u=(u_1,\dots ,u_p)$. Write

$$\begin{array}{rl}& {\int }_0^{{\beta }_{n,I}}\frac{{\partial }^q{N}_n(u_I)}{\partial u_I}d{u}_I\\ & ={\int }_0^{{\beta }_{n,m_1}}\cdots {\int }_0^{{\beta }_{n,m_q}}\frac{{\partial }^q{N}_n(u_I)}{\partial u_{m_1}\cdots \partial u_{m_q}}d{u}_{m_1}\cdots d{u}_{m_q},w_i=x_{in}{x}_{i{m}_1}\cdots x_{i{m}_q}.\end{array}$$

In the following, we will prove that

$$\left|\frac{{\partial }^q{N}_n(u_I)}{\partial u_I}\right|=\left|\sum _{i=1}^n\{{\psi }_i^{(q)}(-x_{in}^T{u}_I;{\mathcal{F}}_{i-1})-{\phi }^{(q)}(-x_{in}^T{u}_I)\}{w}_i\right|=O\left(\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2q}}\right)$$ 4.28

uniformly over $|u|\le p{\delta }_n$.

In fact, let

$$T_n=\sum _{i=1}^n\{{\psi }_i^{(q)}(-x_{in}^T{u}_I;{\mathcal{F}}_{i-1})-{\phi }^{(q)}(-x_{in}^T{u}_I)\}{w}_i$$

and

$$J_k=\sum _{i=1}^n{\mathcal{P}}_{i-k}\{{\psi }_i^{(q)}(-x_{in}^T{u}_I;{\mathcal{F}}_{i-1})-{\phi }^{(q)}(-x_{in}^T{u}_I)\}{w}_i.$$

Then $T_n={\sum }_{k=0}^{\mathrm{\infty }}{J}_k$, and $J_k$ are martingale differences. By the orthogonality of martingale differences and the stationarity of $\{e_i\}$, and Lemma 4.3, we have

$$\begin{array}{rl}{\parallel J_k\parallel }^2& =\sum _{i=1}^n{\parallel {\mathcal{P}}_{i-k}\{{\psi }_i^{(q)}(-x_{in}^T{u}_I;{\mathcal{F}}_{i-1})-{\phi }^{(q)}(-x_{in}^T{u}_I)\}{w}_i\parallel }^2\\ & =\sum _{i=1}^n|w_i{|}^2{\parallel {\mathcal{P}}_0\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})-{\phi }^{(q)}(-x_{kn}^T{u}_I)\}\parallel }^2.\end{array}$$ 4.29

By Lemma 4.3, ${\psi }_i(\cdot ;{\mathcal{F}}_{i-1})\in C^l,l\ge 0$ and the $c_r$-inequality, for $k\ge 0$, we have

$$\begin{array}{rl}& {\parallel {\mathcal{P}}_0\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})-{\phi }^{(q)}(-x_{kn}^T{u}_I)\}\parallel }^2\\ & \le \parallel E\{[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})-{\phi }^{(q)}(-x_{kn}^T{u}_I)]|{\mathcal{F}}_0\}\\ & -E\{[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})-{\phi }^{(q)}(-x_{kn}^T{u}_I)]|{\mathcal{F}}_0^{\ast }\}{\parallel }^2+R_k^2\\ & \le 2{\parallel E\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})|{\mathcal{F}}_0\}-E\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_0^{\ast }\}\parallel }^2\\ & +2{\parallel E\{{\phi }^{(q)}(-x_{kn}^T{u}_I)|{\mathcal{F}}_0\}-E\{{\phi }^{(q)}(-x_{kn}^T{u}_I)|{\mathcal{F}}_0^{\ast }\}\parallel }^2+R_k^2\\ & \le 2{\parallel E\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1})|{\mathcal{F}}_0\}-E\{{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_0^{\ast }\}\parallel }^2\\ & +2|E{\psi }^{(q)}(e_k-x_{kn}^T{u}_I)-E{\psi }^{(q)}(e_k^{\ast }-x_{kn}^T{u}_I){|}^2+R_k^2,\end{array}$$ 4.30

where

$$\begin{array}{rl}{R}_k^2=& \parallel E\{[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })-{\phi }^{(q)}(-x_{kn}^T{u}_I)]|{\mathcal{F}}_{-1}\}\\ & -E\{[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })-{\phi }^{(q)}(-x_{kn}^T{u}_I)]|{\mathcal{F}}_0\}{\parallel }^2.\end{array}$$

Note that $E{\psi }^{(q)}(e_i+\delta )=\frac{d^{q}E\psi (e_i+t)}{d{t}^q}{|}_{t=\delta }$, we have

$$\begin{array}{rl}{R}_k^2\le & {\parallel E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_{-1}]-E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_0]\parallel }^2\\ & +{\parallel E[{\phi }^{(q)}(-x_{kn}^T{u}_I)|F_{-1}]-E[{\phi }^{(q)}(-x_{kn}^T{u}_I)|{\mathcal{F}}_0]\parallel }^2\\ =& {\parallel E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_{-1}]-E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_0]\parallel }^2\\ & +{\parallel E{\psi }_k^{(q)}(e_k-x_{kn}^T{u}_I)-E{\psi }_k^{(q)}(e_k-x_{kn}^T{u}_I)\parallel }^2\\ =& {\parallel E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_{-1}]-E[{\psi }_k^{(q)}(-x_{kn}^T{u}_I;{\mathcal{F}}_{k-1}^{\ast })|{\mathcal{F}}_0]\parallel }^2.\end{array}$$ 4.31

By the conditions (A6), (A7) and (4.29)–(4.31), we have

$$\begin{array}{rl}\parallel T_n\parallel & =\sqrt{\sum _{i=1}^n|w_i{|}^2}\sum _{k=0}^{\mathrm{\infty }}\parallel J_k\parallel =O\left(\sqrt{\sum _{i=1}^n|w_i{|}^2}\right)\\ & =O\left(\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2q}}\right).\end{array}$$

Let $|u|\le p{\delta }_n$. By ${max}_{1\le i\le n}|x_{in}u|\le p{\delta }_n{r}_n\to 0$. Note that ${\delta }_n\to \mathrm{\infty }$ and ${\delta }_n{r}_n\to 0$. By (4.28), we have

$$\begin{array}{rl}\parallel \underset{|{\beta }_n|\le {\delta }_n}{sup}{\int }_0^{{\beta }_{n,I}}\left|\frac{{\partial }^p{N}_n(u_I)}{\partial u_I}\right|d{u}_I\parallel & \le \parallel {\int }_{-{\delta }_n}^{{\delta }_n}\cdots {\int }_{-{\delta }_n}^{{\delta }_n}\left|\frac{{\partial }^p{N}_n(u_I)}{\partial u_I}\right|d{u}_I\parallel \\ & \le {\int }_{-{\delta }_n}^{{\delta }_n}\cdots {\int }_{-{\delta }_n}^{{\delta }_n}\parallel \frac{{\partial }^p{N}_n(u_I)}{\partial u_I}\parallel d{u}_I\\ & =O\left({\delta }_n^q\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2q}}\right)=O\left({\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}\right).\end{array}$$ 4.32

Since

$$N_n({\beta }_n)-N_n(0)=\sum _{I\in \{1,2,\dots ,p\}}{\int }_0^{{\beta }_{n,I}}\frac{{\partial }^{|I|}{N}_n(u_I)}{\partial u_I}d{u}_I,$$ 4.33

the result (4.27) follows from (4.32) and (4.33). □

Lemma 4.5

Let ${\pi }_i,i\ge 1$ be a sequence of bounded positive numbers, and let there exist a constant $c_0\ge 1$ such that ${max}_{1\le i\le 2^d}{\pi }_i\le c_0{min}_{1\le i\le 2^d}{\pi }_i$ holds for all large n. And let ${\omega }_d=2c_0{\pi }_{2^d}$ and $q>3/2$. Assume that (A5) and ${\tilde{r}}_n=O(\sqrt{n})$ hold. Then as $d\to \mathrm{\infty }$

$$\underset{|{\beta }_n|\le {\omega }_d}{sup}\underset{n<2^d}{max}|{\tilde{M}}_n(\beta )-{\tilde{M}}_n(0)|=O_p(\sqrt{{\tilde{\tau }}_{2^d}({\omega }_d)}{d}^q+2^{-5d/2}),$$

where ${\tilde{M}}_n(\beta )={\sum }_{i=1}^n\{\psi (e_i-x_i^T\beta )-E(\psi (e_i-x_i^T\beta )|{\mathcal{F}}_{i-1})\}{x}_i$.

Proof

Let

$$\begin{array}{rl}& {\mu }_n={(logn)}^{q-1},{\tilde{\varphi }}_n=2{\mu }_{2^d}\sqrt{{\tilde{\tau }}_{2^d}({\omega }_d)}log\left(2^d\right),{\tilde{t}}_{2^d}={\mu }_{2^d}\sqrt{{\tilde{\tau }}_{2^d}({\omega }_d)}/log{\mu }_{2^d},\\ & {\tilde{\mu }}_{2^d}={\tilde{t}}_{2^d}^2,{\tilde{\eta }}_i(\beta )=(\psi (e_i-x_i^T\beta )-\psi (e_i))x_i,{\tilde{T}}_{2^d}=\underset{1\le i\le 2^d}{max}\underset{|{\beta }_n|\le {\omega }_d}{sup}|{\tilde{\eta }}_i(\beta )|\end{array}$$

and

$${\tilde{U}}_{2^d}=\sum _{i=1}^{2^d}E\{{[\psi (e_i+|x_i|{\omega }_d)-\psi (e_i-|x_i|{\omega }_d)]}^2|{\mathcal{F}}_{i-1}\}|x_i{|}^2.$$

Since $q>3/2$ and $2(q-1)>1,{\sum }_{d=2}^{\mathrm{\infty }}{({\mu }_{2^d}^{-1}log{\mu }_{2^d})}^2<\mathrm{\infty }$. By the argument of Lemma 4.2 and the Borel–Cantelli lemma, we have

$$P({\tilde{T}}_{2^d}\ge {\tilde{t}}_{2^d},i.o.)=0\text{and}P({\tilde{U}}_{2^d}\ge {\tilde{u}}_{2^d},i.o.)=0.$$ 4.34

Similar to the proof of (4.12), we have

$$\begin{array}{rl}& P\left\{\underset{k\le 2^d}{max}\right|{\tilde{M}}_{k,j,d}(\beta )-M_{k,j,d}(0)|\ge 2{\tilde{\varphi }}_{2^d},{\tilde{T}}_{2^d}\le {\tilde{t}}_{2^d},{\tilde{U}}_{2^d}\le {\tilde{u}}_{2^d}\}\\ & =O(exp(-\frac{{\tilde{\varphi }}_{2^d}^2}{4{\tilde{t}}_{2^d}{\tilde{\varphi }}_{2^d}+2{\tilde{u}}_{2^d}})).\end{array}$$ 4.35

Let $l=n^{8d}$ and $K_l=\{(k_1/l,\dots ,k_p/l):k_i\in Z,|k_i|\le n^{9d}\}$. Then $\mathrm{\#}{K}_l={(2n^{9d}+1)}^p$. By (4.34) and (4.35), for $\mathrm{\forall }\varsigma >1$, we have

$$P\left\{\underset{\beta \in K_l}{sup}\right|{\tilde{M}}_{k,j,d}(\beta )-M_{k,j,d}(0)|\ge 2{\tilde{\varphi }}_{2^d},{\tilde{T}}_{2^d}\le {\tilde{t}}_{2^d},{\tilde{U}}_{2^d}\le {\tilde{u}}_{2^d}\}=O\left(n^{-\varsigma dp}\right).$$ 4.36

Therefore,

$$P\left\{\underset{\beta \in K_l}{sup}\right|{\tilde{M}}_{k,j,d}(\beta )-M_{k,j,d}(0)|\ge 2{\tilde{\varphi }}_{2^d},\mathrm{i}.\mathrm{o}.\}\to 0,n\to \mathrm{\infty }.$$ 4.37

Since ${\tilde{r}}_n=O(\sqrt{n})$ and ${max}_{1\le i\le 2^d}|x_i^T(\beta -{\langle \beta \rangle }_{l,d})|=O(2^{2^d}{l}^{-1}),C{l}^{-1}V$ in (4.17) can be replaced by $C{l}^{-1}{2}^{2^d}V$, and the lemma follows from $P(V_{2^d}\ge 2^{5d},\mathrm{i}.\mathrm{o}.)=0$. □

Lemma 4.6

Let ${\pi }_i,i\ge 1$ be a sequence of bounded positive numbers, and let there exist a constant $c_0\ge 1$ such that ${max}_{1\le i\le 2^d}\pi \le c_0{min}_{1\le i\le 2^d}{\pi }_i$ and ${\pi }_n=o(n^{-1/2}{(logn)}^2)$ hold for all large n. And let ${\omega }_d=2c_0{\pi }_{2^d}$. Assume that (A6), (A7) and ${\tilde{r}}_n=O(\sqrt{n}{(logn)}^{-2})$ hold. Then

$$\parallel \underset{|\beta |\le {\pi }_n}{sup}|{\tilde{N}}_n(\beta )-{\tilde{N}}_n(0)|\parallel =O\left(\sqrt{\sum _{i-1}^n|x_i{|}^4}{\pi }_n\right),$$ 4.38

and, as $d\to \mathrm{\infty }$, for any $\upsilon >0$,

$$\underset{|\beta |\le {\omega }_d}{sup}\underset{n<2^d}{max}|{\tilde{N}}_n(\beta )-{\tilde{N}}_n(0){|}^2=o_{a.s.}(\sum _{i=0}^{2^d}|x_i{|}^4{\omega }_d^2{d}^5{(logd)}^{1+\upsilon }),$$ 4.39

where ${\tilde{N}}_n(\beta )={\sum }_{i=1}^n\{{\psi }_1(-x_i^T\beta ;{\mathcal{F}}_{i-1})-\phi (-x_i^T\beta )\}{x}_n$.

Proof

Let $Q_{n,j}(\beta )={\sum }_{i=1}^n{\psi }_1(-x_i^T\beta ;F_{i-1})x_{ij},i\le j\le p$, and

$$S_n(\beta )=Q_{n,j}(\beta )-Q_{n,j}(0).$$ 4.40

Note that

$${\pi }_n{\tilde{r}}_n=o\left(n^{-1/2}{(logn)}^2\right)O\left(\sqrt{n}{(logn)}^{-2}\right)=o(1).$$ 4.41

It is easy to see that the argument in the proof of Lemma 4.4 implies that there exists a positive constant $C<\mathrm{\infty }$ such that

$$E\left\{\right|\underset{|\beta |\le {\omega }_d}{sup}|S_n(\beta )-S_{n^{\prime }}(\beta )|{|}^2\}\le C\sum _{q=1}^p{\omega }_d^{2q}\sum _{i=n^{\prime }+1}^n|x_i{|}^2+2q$$ 4.42

holds uniformly over $1\le n^{\prime }<n\le 2^d$. Therefore (4.38) holds.

Let $\mathrm{\Lambda }={\sum }_{r=0}^d{\mu }_r^{-1}$, where

$${\mu }_r={\left\{\sum _{m=1}^{2^{d-r}}{\parallel \underset{|\beta |\le {\omega }_d}{sup}|S_{2^{r}m}(\beta )-S_{2^r(m-1)}(\beta )|\parallel }^2\right\}}^{-1/2}.$$ 4.43

For a positive integer $k\le 2^d$, write its dyadic expansion $k=2^{r_1}+\cdots +2^{r_j}$, where $0\le r_j<\cdots <r_1\le d$, and $k(i)=2^{r_1}+\cdots +2^{r_i}$. By the Schwartz inequality, we have

$$\begin{array}{rl}\underset{|\beta |\le {\omega }_d}{sup}|S_k(\beta ){|}^2& \le {\left\{\sum _{i=1}^j\underset{|\beta |\le {\omega }_d}{sup}\right|S_{k(i)}(\beta )-S_{k(i-1)}(\beta )\left|\right\}}^2\\ & ={\{\sum _{i-1}^j{\mu }_{r_i}^{-1/2}\cdot {\mu }_{r_i}^{1/2}\underset{|\beta |\le {\omega }_d}{sup}|S_{k(i)}(\beta )-S_{k(i-1)}(\beta )\left|\right\}}^2\\ & \le \sum _{i-1}^j{\mu }_{r_i}^{-1}\sum _{i-1}^j{\mu }_{r_i}\underset{|\beta |\le {\omega }_d}{sup}|S_{k(i)}(\beta )-S_{k(i-1)}(\beta ){|}^2\\ & \le \mathrm{\Lambda }\sum _{i-1}^j{\mu }_{r_i}\sum _{m=1}^{2^{d-\eta }}\underset{|\beta |\le {\omega }_d}{sup}|S_{2^{\eta }m}(\beta )-S_{2^{\eta }(m-1)}(\beta ){|}^2\\ & \le \mathrm{\Lambda }\sum _{r=0}^d{\mu }_r\sum _{m=1}^{2^{d-r}}\underset{|\beta |\le {\omega }_d}{sup}|S_{2^{r}m}(\beta )-S_{2^r(m-1)}(\beta ){|}^2.\end{array}$$ 4.44

Thus

$$\begin{array}{rl}\parallel \underset{n\le 2^d}{max}\underset{|\beta |\le {\omega }_d}{sup}|S_n(\beta )|\parallel & =\parallel \sum _{n=1}^{2^d}\underset{|\beta |\le {\omega }_d}{sup}|S_n(\beta )|\parallel \\ & \le \sum _{n=1}^{2^d}\parallel \underset{|\beta |\le {\omega }_d}{sup}|S_n(\beta )|\parallel \le \sum _{n=1}^{2^d}{\left\{E\right|\underset{|\beta |\le {\omega }_d}{sup}|S_n(\beta )|{|}^2\}}^{1/2}\\ & \le \sum _{r=0}^d{\left\{E\right|\mathrm{\Lambda }\sum _{r=0}^d{\mu }_r\sum _{m=1}^{2^{d-r}}\left|\underset{|\beta |\le {\omega }_d}{sup}\right|S_{2^{r}m}(\beta )-S_{2^r(m-1)}(\beta )\left|{|}^2\right|\}}^{1/2}\\ & \le \sum _{r=0}^d{\left\{\mathrm{\Lambda }\sum _{r=0}^d{\mu }_r\sum _{m=1}^{2^{d-r}}{\parallel \underset{|\beta |\le {\omega }_d}{sup}|S_{2^{r}m}(\beta )-S_{2^r(m-1)}(\beta )|\parallel }^2\right\}}^{1/2}\\ & \le \sum _{r=0}^d{\left\{\mathrm{\Lambda }\sum _{r=0}^d{\mu }_r{\mu }_R^{-2}\right\}}^{1/2}=\sum _{r=0}^d{\left\{\mathrm{\Lambda }\sum _{r=0}^d{\mu }_r^{-1}\right\}}^{1/2}=d\mathrm{\Lambda }.\end{array}$$ 4.45

Since $\upsilon >0$ and ${\omega }_d^{2q}{\sum }_{i=1}^{2^d}{|x_i|}^{2+2q}=O({\omega }_d^2{\sum }_{i=1}^{2^d}{|x_i|}^4)$, (4.42) implies that

$$\sum _{d=2}^{\mathrm{\infty }}\frac{{\parallel {max}_{n\le 2^d}{sup}_{|\beta |\le {\omega }_d}|S_n(\beta )|\parallel }^2}{{\omega }_d^2{\sum }_{i=1}^{2^d}|x_i{|}^4{d}^5{(logd)}^{1+\upsilon }}=\sum _{d=2}^{\mathrm{\infty }}\frac{O(d^2{(d+1)}^2)}{d^5{(logd)}^{1+\upsilon }}<\mathrm{\infty }.$$ 4.46

By the Borel–Cantelli lemma, (4.39) follows from (4.46). □

Lemma 4.7

Under the conditions of Theorem 2.2, we have:

1. ${sup}_{|\beta |\le b_n}|{\tilde{K}}_n(\beta )-{\tilde{K}}_n(0)|=O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}})$;

2. for and $\upsilon >0,{\tilde{K}}_n(0)=O_{a.s.}(h_n)$, where $h_n=n^{1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon /4}$.

Proof

Observe that ${\tilde{K}}_n(\beta )={\tilde{M}}_n(\beta )+{\tilde{N}}_n(\beta )$. Since $n^{-5/2}=o(B_{\tilde{n}})$, (1) follows from Lemma 4.5 and 4.6. □

As with the argument in (4.29), we have ${\tilde{K}}_n(0)=O(\sqrt{n})$.

Proof of Theorem 2.1

Observe that

$$\begin{array}{rl}{K}_n({\beta }_n)=& \sum _{i=1}^n\psi (e_i-x_{in}^T{\beta }_n)x_{in}-E\left(\sum _{i=1}^n\psi (e_i-x_{in}^T{\beta }_n)x_{in}\right)\\ =& \sum _{i=1}^n\{\psi (e_i-x_{in}^T{\beta }_n)-E(\psi (e_i-x_{in}^T{\beta }_n)|{\mathcal{F}}_{i-1})\}{x}_{in}\\ & +\sum _{i=1}^n\{E(\psi (e_i-x_{in}^T{\beta }_n)|{\mathcal{F}}_{i-1})-E\psi (e_i-x_{in}^T{\beta }_n)\}{x}_{in}\\ =& M_n({\beta }_n)+N_n({\beta }_n).\end{array}$$ 4.47

By (4.47), Lemma 4.2 and Lemma 4.4, we have

$$\begin{array}{rl}\underset{|{\beta }_n|\le {\delta }_n}{sup}|K_n({\beta }_n)-K_n(0)|& \le \underset{|{\beta }_n|\le {\delta }_n}{sup}|M_n({\beta }_n)-M_n(0)|+\underset{|{\beta }_n|\le {\delta }_n}{sup}|N_n({\beta }_n)-N_n(0)|\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn+n^{-3})+O\left(\sqrt{\sum _{i=1}^n|x_{in}{|}^4}{\delta }_n\right)\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}).\end{array}$$ 4.48

This completes the proof of Theorem 2.1. □

Proof of Corollary 2.1

Take an arbitrary sequence ${\delta }_n\to \mathrm{\infty }$, which satisfies the assumption of Theorem 2.1. Note that

$$K_n(0)=\sum _{i=1}^n\psi (e_i)x_{in}-E(\sum _{i=1}^n\psi (e_i)x_{in})=\sum _{i=1}^n\psi (e_i)x_{in}$$ 4.49

and

$$\begin{array}{rl}{K}_n({\hat{\beta }}_n)& =\sum _{i=1}^n\psi (e_i-x_{in}^T{\hat{\beta }}_n)x_{in}-E\left(\sum _{i=1}^n\psi (e_i-x_{in}^T{\hat{\beta }}_n)x_{in}\right)\\ & =\sum _{i=1}^n\psi (y_i-x_{in}^T{\hat{\beta }}_n)x_{in}-\sum _{i=1}^n\phi (-x_{in}^T{\hat{\beta }}_n)x_{in}\\ & =-\sum _{i=1}^n\phi (-x_{in}^T{\hat{\beta }}_n)x_{in}+O_P(r_n)\end{array}$$ 4.50

for $|{\hat{\beta }}_n|\le {\delta }_n$. By Theorem 2.1 and (4.49), we have

$$K_n({\hat{\beta }}_n)=\sum _{i=1}^n\psi (e_i)x_{in}+O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}).$$ 4.51

By (4.50) and (4.51), we have

$$\begin{array}{rl}& -\sum _{i=1}^n\phi (-x_{in}^T{\hat{\beta }}_n)x_{in}+O_p(r_n)\\ & =\sum _{i=1}^n\psi (e_i)x_{in}+O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}).\end{array}$$ 4.52

By (4.52), $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$ as $t\to 0$, and ${\sum }_{i=1}^n{x}_{in}{x}_{in}^T=I_p$, we have

$$\begin{array}{rl}& -\sum _{i=1}^n\{(-x_{in}^T{\hat{\beta }}_n){\phi }^{\prime }(0)+O\left({(-x_{in}^T{\hat{\beta }}_n)}^2\right)\}{x}_{in}-\sum _{i=1}^n\psi (e_i)x_{in}\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4})-O_p(r_n)\end{array}$$

and

$$\begin{array}{rl}& \sum _{i=1}^n{x}_{in}{x}_{in}^T{\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}\\ & =-\sum _{i=1}^{n}O\left({(-x_{in}^T{\hat{\beta }}_n)}^2\right)x_{in}+O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4})-O_p(r_n).\end{array}$$

Namely

$$\begin{array}{rl}& {\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn)+O_p({\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}+\sum _{i=1}^n{(-x_{in}^T{\hat{\beta }}_n)}^2{x}_{in}+r_n)\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn)+O_p({\delta }_n\sqrt{\sum _{i=1}^n|x_{in}{|}^4}+|{\hat{\beta }}_n{|}^2\sum _{i=1}^n|x_{in}{|}^3+r_n)\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn)+O_p({\delta }_n{r}_n+{\delta }_n^2{r}_n+r_n)\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n^2{r}_n).\end{array}$$ 4.53

By $m(t)=O({|t|}^{\lambda })(t\to 0)$ for some $\lambda >0$, we have

$${\tau }_n({\delta }_n)=2\sum _{i=1}^n|x_{in}{|}^2{(|x_{in}|{\delta }_n)}^{2\lambda }=2{\delta }_n^{2\lambda }\sum _{i=1}^n|x_{in}{|}^{2+2\lambda }.$$ 4.54

Then it follows from (4.53) and (4.54) that

$$\begin{array}{rl}& {\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}\\ & =O_p(\sqrt{{\tau }_n({\delta }_n)}logn+{\delta }_n^2{r}_n)\\ & =O_p(\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2\lambda }}{\delta }_n^{\lambda }logn+{\delta }_n^2{r}_n)\end{array}$$ 4.55

for any ${\delta }_n\to \mathrm{\infty }$, which implies

$${\phi }^{\prime }(0){\hat{\beta }}_n-\sum _{i=1}^n\psi (e_i)x_{in}=O_p(\sqrt{\sum _{i=1}^n|x_{in}{|}^{2+2\lambda }}logn+r_n).$$ 4.56

 □

Proof of Theorem 2.2

By Lemma 4.7, we have Theorem 2.2. □

Proof of Corollary 2.2

(1) By Lemma 4.7, we have

$$\underset{|{\beta }_n|\le b_n}{sup}|{\tilde{K}}_n({\beta }_n)|\le \underset{|{\beta }_n|\le b_n}{sup}|{\tilde{K}}_n({\beta }_n)-{\tilde{K}}_n(0)|+{\tilde{K}}_n(0)=O_{a.s.}(L_{\tilde{n}}+B_{\tilde{n}}+h_n),$$ 4.57

where $b_n=n^{-1/2}{(logn)}^{3/2}{(loglogn)}^{1/2+\upsilon }$. Let

$${\mathrm{\Theta }}_n(\beta )=\sum _{i=1}^n[\rho (e_i-x_i^T\beta )-\rho (e_i)]$$ 4.58

and

$$A_n(\beta )=-\sum _{i=1}^n{\int }_0^1\phi (-x_i^T\beta )x_i^T\beta dt.$$ 4.59

Note that

$$\rho (e_i)-\rho (e_i-x_i^T\beta )={\int }_0^1\psi (e_i-x_i^T\beta )x_i^T\beta dt.$$ 4.60

By (4.57)–(4.60), we have

$$\begin{array}{rl}\underset{|{\beta }_n|\le b_n}{sup}|{\mathrm{\Theta }}_n(\beta )-A_n(\beta )|& =\underset{|{\beta }_n|\le b_n}{sup}\left|\sum _{i=1}^n{\int }_0^1[\psi (e_i-x_i^T\beta )-\phi (-x_i^T\beta )]x_i^T\beta dt\right|\\ & =\underset{|{\beta }_n|\le b_n}{sup}|{\int }_0^1{\tilde{K}}_n(\beta t)\beta dt|\\ & =O_{a.s.}((L_{\tilde{n}}+B_{\tilde{n}}+h_n)b_n).\end{array}$$ 4.61

It is easy to show that $b_n^3{\sum }_{i=1}^n{|x_i|}^3=O(n{\tilde{r}}_n)b_n^3=o(n{b}_n^2)$. By $\phi (t)=t{\phi }^{\prime }(0)+O(t^2)$, we have

$$\begin{array}{rl}\underset{|{\beta }_n|=b_n}{inf}{A}_n(\beta )& =\underset{|{\beta }_n|=b_n}{inf}\{-\sum _{i=1}^n{\int }_0^1\phi (-x_i^T\beta )x_i^T\beta dt\}\\ & =\underset{|{\beta }_n|=b_n}{inf}\{-\sum _{i=1}^n{\int }_0^1[\phi (0)+{\phi }^{\prime }(0)(-x_i^T\beta )+O\left({(-x_i^T\beta )}^2\right)]x_i^T\beta dt\}\\ & =\underset{|{\beta }_n|=b_n}{inf}\left\{\sum _{i=1}^n[\frac{1}{2}{\phi }^{\prime }(0){(-x_i^T\beta )}^2{|}_0^1-\frac{1}{3}O\left({(-x_i^T\beta )}^3\right){|}_0^1]\right\}\\ & =\frac{1}{2}{\phi }^{\prime }(0)\sum _{i=1}^n{x}_i^T{x}_i{b}_n^2-\frac{1}{3}\underset{|{\beta }_n|=b_n}{inf}\left\{b_n^2\sum _{i=1}^{n}O\left(x_i^T{x}_i\right)x_i^T\beta \right\}\\ & \ge \frac{1}{2}{\phi }^{\prime }(0)S_n{b}_n^2-\frac{1}{3}{b}_n^2\sum _{i=1}^n|x_i{|}^3{b}_{n}O(1)\\ & \ge \frac{1}{6}{\phi }^{\prime }(0)n{b}_n^2\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}inf}{\lambda }_n/n.\end{array}$$ 4.62

By $m(t)=O(\sqrt{n})$ as $t\to 0$, we have $(L_{\tilde{n}}+B_{\tilde{n}}+h_n)b_n=o(n{b}_n^2)$. Thus

$$\begin{array}{rl}\underset{|{\beta }_n|=b_n}{inf}{\mathrm{\Theta }}_n(\beta )& \ge \underset{|{\beta }_n|=b_n}{inf}{A}_n(\beta )-\underset{|{\beta }_n|\le b_n}{sup}|{\mathrm{\Theta }}_n(\beta )-A_n(\beta )|\\ & \ge \frac{1}{6}{\phi }^{\prime }(0)n{b}_n^2\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}inf}{\lambda }_n/n+O_{a.s.}((L_{\tilde{n}}+B_{\tilde{n}}+h_n)b_n)\\ & \ge \frac{1}{4}{\phi }^{\prime }(0)n{b}_n^2\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}inf}{\lambda }_n/n,\text{a.s.}\end{array}$$ 4.63

By the convexity of the function ${\mathrm{\Theta }}_n(\cdot )$, we have

$$\begin{array}{rl}& \{\underset{|{\beta }_n|\ge b_n}{inf}{\mathrm{\Theta }}_n(\beta )\ge \frac{1}{4}{\phi }^{\prime }(0)n{b}_n^2\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}inf}{\lambda }_n/n\}\\ & =\{\underset{|{\beta }_n|=b_n}{inf}{\mathrm{\Theta }}_n(\beta )\ge \frac{1}{4}{\phi }^{\prime }(0)n{b}_n^2\underset{n\to \mathrm{\infty }}{lim\hspace{0.17em}inf}{\lambda }_n/n\}.\end{array}$$ 4.64

Therefore the minimizer ${\hat{\beta }}_n$ satisfies ${\hat{\beta }}_n=O_{a.s.}(b_n)$.

(2) Let $|{\hat{\beta }}_n|\le b_n$. By a Taylor expansion, we have

$$\begin{array}{rl}-\sum _{i=1}^n\phi (-x_i^T\beta )x_i& =\sum _{i=1}^n[{\phi }^{\prime }(0)x_i^T\beta +O\left(\right|x_i^T\beta {|}^2)]x_i\\ & ={\phi }^{\prime }(0){\mathrm{\Sigma }}_n\beta +O(b_n^2\sum _{i=1}^n|x_i{|}^3).\end{array}$$ 4.65

Therefore (2) follows from Theorem 2.2 and (1). □

Authors’ contributions

The author organized and wrote this paper. Further he examined all the steps of the proofs in this paper. The author read and approved the final manuscript.

Competing interests

The author declares to have no competing interests.