Partially Linear Varying Coefficient Models Stratified by a Functional Covariate

Abstract

We consider the problem of estimation in semiparametric varying coefficient models where the covariate modifying the varying coefficients is functional and is modeled nonparametrically. We develop a kernel-based estimator of the nonparametric component and a profiling estimator of the parametric component of the model and derive their asymptotic properties. Specifically, we show the consistency of the nonparametric functional estimates and derive the asymptotic expansion of the estimates of the parametric component. We illustrate the performance of our methodology using a simulation study and a real data application.

1 Introduction

Suppose we observe for i = 1, …, n outcome Y_i and covariates X_i, S_i and , where X_i = (X_i1, …, X_ip)^T ∈ ℜ^p, S_i = (S_i1, …, S_iq)^T ∈ ℜ^q and is a functional covariate assumed to belong to a function space . We consider the following partially linear functional varying coefficient model:

$$Y_i=X_i^{\text{T}}{\beta }_0({\mathcal{Z}}_i)+S_i^{\text{T}}{\eta }_0+{\epsilon }_i,$$ (1)

where β₀(t) = [β₀₁(t), …, β_0p(t)]^T with β_0k(·), k = 1, …, p, assumed to be smooth but unknown functionals, η₀ ∈ ℜ^q and the random errors ε_i are independently and identically distributed with E[ε|X, , S] = 0. We also assume the functional covariate is observed on a finite grid 0 ≤ t₁ < … < t_s ≤ 1, without loss of generality.

This model is the functional generalization of the standard partially linear varying coef-ficient model where is typically assumed to be a scalar or a vector (Ahmad et al. 2005). A simple example of such model is when is binary (taking values 0 or 1) indicating a non-exposed or exposed group to some treatment. In that case, β₀( ) has only two values indicating the different effects of X for the two groups. In general, can be thought as a stratification variable that divides the population into several strata and the varying coefficient model assumes that X has different effect on Y in different strata. In other words, it is a means to quantify the interaction between X and . Such varying coefficient models have obtained much attention in the past years due to their usefulness and a fair amount of literature exists when the nonparametrically modeled variable is a vector. For example, Fan and Huang (2005) considered semiparametric varying-coefficient partially linear models and derived asymptotic properties of profile likelihood estimators. For a much more detailed review, readers are directed to Fan and Zhang (2008) and the references therein. However, in this paper, we consider the case where is a functional stratifier and develop an estimation procedure for β₀ and η₀.

Semiparametric models are not well studied when covariates modeled nonparametrically are functional-valued. There is some recent literature on partially linear functional models. Aneiros-Perez and Vieu (2006) considered a special case of model (1) with X_i ≡ 1. Aneiros-Perez and Vieu (2008) considered semi-functional partial linear time series modeling to use a continuous path in the past (used as the functional covariate) to predict future values of the process. Lian (2011) considered functional partial linear models in the sense that the functional covariate is modeled via a linear operator while the other covariate (assumed to be in a vectorial topological space with a semi-metric) is modeled nonparametrically. Ferraty and Romain (2010) also provided detailed discussion about functional partial linear models, their estimation procedure using kernel smoothing and asymptotic properties of the estimates. However, to the best of our knowledge, there is no results on partially linear functional varying-coefficient model where the varying coefficients are functionals of the observed functional covariates and are modeled nonparametrically.

In this note, we extend the work of Aneiros-Perez and Vieu (2006) and also discuss asymptotic expansion and properties of estimators of η₀, which is new when there exist functional covariates that enter a regression model nonparametrically. Section 2 presents our estimation method and relevant asymptotic results. Some simulation results are presented in Section 3 and an application to real data is given in Section 4. The Appendix collects all technical conditions and proofs.

2 Estimation method and asymptotic results

2.1 Estimation of β0(·)

We first discuss about estimation of β(·) for any fixed value of η₀ = η^*. Recall that we assume the covariates to be in a abstract space . Ferraty and Vieu (2004, 2006) argued that the most interesting spaces for functional data modeling are the so called semi-metric spaces. Here, to keep the general formulation, we assume that is a semi-metric space with an associated semi-metric d(·, ·).

We now use Nadaraya-Watson type local constant kernel based estimators for β. Let K(·) denote the kernel function and h be a bandwidth, and define K_h(z) = K (z/h). In general, K(·) should satisfy some usual smoothness conditions, see Appendix for details. Define (η^*) = Y − S^Tη^*. Then, for any given z ∈ , we estimate β₀(z) by minimizing

$$\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{\{{\mathcal{Y}}_i({\eta }^{\ast })-X_i^{\text{T}}\beta (z)\}}^2,$$

with respect to β(·) or equivalently, one solves for β(z),

$$0=\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i\{{\mathcal{Y}}_i({\eta }^{\ast })-X_i^{\text{T}}\beta (z)\}.$$

Of course, the solution has a closed form:

$$\widehat{\beta }(z,{\eta }^{\ast })={\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{X}_i^{\text{T}}\right]}^{-1}\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{\mathcal{Y}}_i({\eta }^{\ast }).$$ (2)

Define β(z, η) = [E(XX^T| = z)]⁻¹E[X(Y − S^T η)| = z]. Note that β₀(z) = β(z, η₀). Suppose the functions , …, ∈ where is a compact subset of such that condition (C. 1) in the Appendix is satisfied. Then we have the following result.

Result 1

Under regularity conditions given in the Appendix, we have that uniformly in z ∈

$$\widehat{\beta }(z,{\eta }_0)-\beta (z,{\eta }_0)=O(h^a)+O\left(\sqrt{\frac{logn}{n\phi (h)}}\right)a.s.,$$

where φ (h) is defined in the Appendix.

The proof of Result 1 is given in the Appendix.

2.2 Estimation of η₀

We estimate η₀ using the profile method, as described below. Recall that for each value of η, we can construct β̂(z, η) as in (2). Then we propose to estimate η₀ by minimizing

$$\sum _{i=1}^n{\{Y_i-S_i^{\text{T}}\eta -X_i^{\text{T}}\widehat{\beta }({\mathcal{Z}}_i,\eta )\}}^2$$

and thereby solving for η

$$0=n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}\{Y_i-S_i^{\text{T}}\eta -X_i^{\text{T}}\widehat{\beta }({\mathcal{Z}}_i,\eta )\},$$

where

$$\begin{array}{l}{\widehat{\beta }}_{\eta }(z)={[{\widehat{\beta }}_{\eta ,1}(z),\dots ,{\widehat{\beta }}_{\eta ,q}(z)]}^{\text{T}};\\ {\widehat{\beta }}_{\eta ,j}(z)=-{\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{X}_i^{\text{T}}\right]}^{-1}\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{S}_{ij}.\end{array}$$

To investigate the asymptotic properties of η̂, we first define β_{η, j} (z) = −[E(XX^T| = z)]⁻¹E[XS_j | = z], and $\stackrel{\sim }{S}=S-E[{SX}^{\text{T}}\mid \mathcal{Z}]{[E({XX}^{\text{T}}\mid \mathcal{Z})]}^{-1}X=S+{\beta }_{\eta }^{\text{T}}(\mathcal{Z})X$. Then we have the following result.

Result 2

Under regularity conditions given in the Appendix, we have that uniformly in z ∈

$$n^{1/2}(\widehat{\eta }-{\eta }_0)=n^{-1/2}{\mathrm{\sum }}^{-1}\sum _{i=1}^n{\stackrel{\sim }{S}}_i{\epsilon }_i+o_p(1)\to \text{Normal}(0,{\sigma }^2{\mathrm{\sum }}^{-1}),$$

where σ² = E(ε²) and Σ = E(S̃S̃^T).

Remark 1

Note that by definition, S̃ does not depend on the true value of η₀, and hence Σ is also independent of η₀. Also it is interesting to note that S̃_j, the j^th element of S̃ for each j, can be thought of the residual of a nonparametric varying coefficient model with S_j, the j^th element of S, as response and X and as covariates. Specifically, if we posit the nonparametric varying coefficient model $S_{ij}=X_i^{\text{T}}{\mathcal{G}}_j({\mathcal{Z}}_i)+{\zeta }_i$, i = 1, …, n, E(ζ_i|X_i, ) = 0, then by similar arguments as in Result 1, (z) gives us consistent estimate of −β _η,j (z), regardless of the correctness of the posited model. Thus S̃ can be estimated by the residuals of the element-wise regressions of S on X and using nonparametric varying coefficient models. Consequently, one can easily estimate Σ using plug-in method where one replaces expectation by the corresponding sample version in the expression of Σ. The error variance σ² can also be estimated by the mean square error of the estimated residuals from the full semiparametric model.

3 Simulation Study

In this section we present a simulation study to evaluate the performance of our method. We consider the model

$$Y_i=X_{i1}{\beta }_1({\mathcal{Z}}_i)+X_{i2}{\beta }_2({\mathcal{Z}}_i)+{\eta }_{00}+S_i{\eta }_{01}+{\epsilon }_i,$$

where we set X_i1 = Normal(0, 1), X_i2 = Normal(0, 1), S_i = Normal(0, 1), and ε_i = Normal(0, 1). The true values of the parameters are η₀ = (η₀₀, η₀₁)^T = (5, 5)^T. The functional covariates are taken to be of the form

$${\mathcal{Z}}_i(t)=a_i{(t-0.5)}^2+b_i$$

with a_i = Normal(0, 4), b_i = Normal(0, 1) and t ∈ [0, 1]. We take for z(t) ε

$$\begin{array}{l}{\beta }_1(z)={\int }_0^1{\{z^{(1)}(t)\}}^{2}dt;\\ {\beta }_2(z)={\int }_0^1\{\mid z^{(1)}(t)\mid \}\text{log}\{\mid z^{(1)}(t)\mid \}dt.\end{array}$$

The functions are observed in a grid of 51 equally spaced points on [0, 1]. We use B-splines to approximate the observed functions and use first derivative based semi-metric. To fit each model, we use k-nearest neighbors type bandwidths which are selected by cross-validation.

To evaluate the performance of the estimator, we set up a test set of functions = {a(t − 0.5)² + 1: −3 ≤ a ≤ 3}. We consider three different sample sizes, n = 200, 400 and 1000, and generate 500 data sets for each case.

For each data set, we estimate β₁(z) and β₂(z) with z ∈ and compare the estimates with the actual values. The results are shown in Figure 1. Plotted are the actual values of the operators with z ∈ and the median of the estimates over all simulated data sets along with 95% point-wise confidence bands. It is evident from the graphs that the estimates are very close to the true functional values for z ∈ with reasonable confidence bands for both the functionals.

Figure 1.

Results for the simulation study. Plotted are the actual values of β₁(z) (1st column) and β₂(z) (2nd column) with z ∈ (dashed line) and the median of the estimates over all simulated data sets (solid line) along with point wise empirical 95% confidence band (dotted line). The top, middle and bottom rows correspond to three different sample sizes, n = 200, 400 and 1000, respectively.

Also, we estimate η₀₀ and η₀₁ for each of the settings. We compute, for j = 0, 1, the scaled empirical bias as average of n^1/2(η̂_0j − η_0j), scaled empirical standard error as the standard deviation of n^1/2(η̂_0j − η_0j), estimated standard error as the average of estimated standard deviation of n^1/2(η̂_0j − η_0j) (using Remark 1) and empirical coverage at 95% nominal level. The results are given in Table 1. It is evident that as n increases, the bias remains close to zero, with reasonable coverage and standard errors. To observe normality of the estimates, we provide Q-Q plot of the estimates of η₀₀ and η₀₁ against normal quantiles when n = 1000 in Figure 2. It is evident that for large sample size, the distribution of estimates behaves like a Gaussian distribution.

Table 1.

Simulation results from Section 3. Presented are the scaled empirical bias (Bias = average of n^1/2(η̂_0j − η_0j)), empirical standard error (Emp.se = standard deviation of n^1/2(η̂_0j − η_0j)), estimated standard error (Est.se = average of estimated standard deviation of n^1/2(η̂_0j − η_0j)) and empirical coverage (at 95% nominal level) of η₀₀ and η₀₁ over 500 simulations.

	Parameter	Bias	Emp.se	Est.se	Emp.cov (%)
n = 200	η00	0.031	1.43	1.22	91.6
	η01	−0.043	1.34	1.18	92.0
n = 400	η00	0.019	1.25	1.09	93.6
	η01	−0.034	1.27	1.09	93.5
n = 1000	η00	0.005	1.12	1.02	94.2
	η01	−0.006	1.13	1.02	94.6

Figure 2.

Results for the simulation study. Displayed are the normal Q-Q plot of estimates of η₀₀ (left) and η₀₁ (right) for n = 1000.

4 Data Example

We demonstrate out method using the Tecator data set available at http://lib.stat.cmu.edu/datasets/tecator. The goal is to predict the fat content of a meat sample on the basis of its near infrared absorbance spectrum. Each sample contains finely chopped pure meat with different moisture, fat and protein contents, which are measured in percent and are determined by analytic chemistry. The functional covariate for each food sample consist of a 100 channel spectrum of absorbances recorded on a Tecator Infratec Food and Feed Analyzer working in the wavelength range 850–1050 nm by the near infrared transmission (NIT) principle. See Ferraty and Vieu (2006) for a detailed discussion of the data set.

In this analysis, we demonstrate our method using a subset of the full data. We have for i = 1, …, 215 samples where our response variable Y_i is the fat content in the sample. We denote the protein and moisture content by X₁ and X₂, respectively, and denote the functional covariate by . We consider various (varying coefficient) models. To evaluate the performance of the models, we divide the data set into a training data consisting of 172 samples and a test set with 43 samples as suggested in the data documentation given at the above mentioned link. We use the prediction mean squared error

$$\mathit{PMSE}=\sum _{i=1}^{43}{({\widehat{Y}}_i-Y_i)}^2/43$$

as our evaluation criterion. To fit each model, we use k-nearest neighbors type bandwidths which are selected by cross-validation over the training sample. We use derivative based semi-metrics d_k(f, g) = [∫{f ^(k)(x) − g^(k)(x)}²dx]^1/2 for k = 0, 1, 2. The results are displayed in Table 2.

Table 2.

Prediction mean squared errors for the Tecator data set for different type of varying coefficient models. Columns denoting PMSE(k) display PMSE values when the semi-metric d_k is used. In each model, the best PMSE value is given in bold face. The overall best PMSE is underlined and bold faced.

Model	PMSE(2)	PMSE(1)	PMSE(0)
Y = β0( ) + X1β1( ) + ε	2.72	5.55	8.82
Y = β0( ) + X2β2( ) + ε	1.80	1.41	2.28
Y = β0( ) + X1β1 + X2β2 + ε	1.05	0.80	2.03
Y = β0( ) + X1β1( ) + X2β2 + ε	1.29	0.88	1.58
Y = β0( ) + X1β1 + X2β2( ) + ε	1.30	0.40	1.47
Y = β0( ) + X1β1( ) + X2β2( ) + ε	0.99	0.56	1.48

It is evident that the model with linear effect of X₁ and varying coefficient for X₂ performs best among the all considered models. It is interesting to note that Aneiros-Perez and Vieu (2006) found that the Semi-functional partial linear model with both X₁ and X₂ linear was best among their models. From our results, it appears that allowing X₂ to have a varying coefficient captured the data better. In fact, the model with both X₁ and X₂ having varying coefficient performs competitively as well.

We emphasize that the example provided in this section is not meant to be a full case study or analysis of the data, but a demonstrative example that shows the performance of out methods and illustrates the fact that the partially linear functional varying coefficient models are competitive for this kind of data sets.

A Appendix

A.1 Conditions and Assumptions

We use the following assumptions for the results stated in this paper. These assumptions are usual in nonparametric functional modeling, see for example Ferraty and Vieu (2006) and Aneiros-Perez and Vieu (2006).

• (C. 1)The functions , …, ∈ where is a compact subset of such that ${\cup }_{k=1}^{{\tau }_n}\mathcal{B}(z_k,{\ell }_n)$, with ${\tau }_n{\ell }_n^{\gamma }=c_0$ for some real positive constants γ, c₀.
• (C.K)K(·) has support [0, 1], is Lipschitz continuous and is strictly decreasing. Also, there exists c_K > 0 such that for all t ∈ [0, 1], −K′(t) > c_K.
• (C. θ)There exist constants c and a so that |f (z₁) − f (z₂)| ≤ cd(z₁, z₂)^a for all z₁, z₂ ∈ with f ∈ {β₁, …, β_p, βη_,1, …, β_η,q, g₁₁, …, g_pp, w₁₁, …, w_pq}, where g_{j k}(z) = E(X_j X_k| = z) and w_{j k}(z) = E(X_j S_k| = z).
• (C.Y)For some β ≥ 2, E|Y |^β < ∞.
• (C. 2)
  There exists a function φ(h) with ${\int }_0^1\phi (ht)dt/\phi (h)>{\theta }_2>0$ for some θ₂ > 0 such that for all z ∈ , there are constants c₁ and c₂ such that

  $$c_1\phi (h)\le P[\mathcal{Z}\in B(z,h)]\le c_2\phi (h).$$
• (C.X)E(XX^T| ) is finite positive definite. Also for all m ≥ 2, E(|X_j X_k|^m| = z) < σ_1m(z) < ∞, where σ_1m(·) is continuous at z.
• (C.S)E(SS^T| ) is finite positive definite. Also for all m ≥ 2, E(|X_j S_k|^m| = z) < σ_2m(z) < ∞, where σ_2m(·) is continuous at z.
• (C.ε)σ² = E(ε²) > 0 and E|ε|^r < ∞ for some r ≥ 3.
• (C.h)nh^4a → 0 and φ (h) ≥ n^(2/r)+b−1/(log(n))² for large enough n and some constant b > 0 such that (2/r) + b > 1/2

Remark 2

Note that the assumption (C. 1) describes the structure of the subset containing the functional covariates. As noted in Ferraty and Vieu (2008), this assumption is trivially satisfied in standard non-parametric problems with vector-valued covariates, but this is not necessarily true for any general abstract semi-metric space. Thus it is necessary to assume that the compact subset can be written such that (C. 1) holds. To address this issue, Ferraty et al. (2010) proposed an alternative condition based on Kolmogorov’s ε-entropy (see their definition 1 in Section 2). Specifically, let us denote the Kolmogorov’s ε-entropy of by Ψ (ε). Ferraty et al. (2010) proposed to impose conditions on Ψ (log(n)/n), see for example their conditions (H5) and (H6). In this paper, we will utilize the condition (C. 1) to prove our results. However, we conjecture that it is possible to prove similar results using these alternative entropy based conditions as well. We do not take-up this problem in the present paper but it is certainly an interesting open-problem.

A.2 Proof of Result 1

We use the following two results to prove Result 1. Denote ${\mathcal{Y}}_i={\mathcal{Y}}_i({\eta }_0)=Y_i-S_i^{\text{T}}{\eta }_0$.

Lemma 1

Under regularity conditions, uniformly in z ∈ ,

$${\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}\right]}^{-1}\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{X}_i^{\text{T}}\right]\to E({XX}^{\text{T}}\mid \mathcal{Z}=z)\text{a}.\text{s}.$$

Lemma 2

Under regularity conditions, uniformly in z ∈ ,

$${\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}\right]}^{-1}\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{\mathcal{Y}}_i\right]=E[{XX}^{\text{T}}\mid \mathcal{Z}=z]\beta (z)+O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)\text{a}.\text{s}.$$

Now, Using Lemma 1, Lemma 2 and positive definiteness of E[XX^T| = z], it is easy to derive that uniformly in z ∈

$$\widehat{\beta }(z)-\beta (z)=O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)a.s.$$

A.2.1 Proof of Lemma 1

Using results in Ferraty and Vieu (2004), see proof of their Lemma 3.1, we see that for j, k = 1, …, p,

$$\underset{z\in \mathcal{C}}{sup}\left|{\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}\right]}^{-1}\left[\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_{ij}{X}_{ik}\right]-E(X_j{X}_k\mid \mathcal{Z}=z)\right|=O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)\text{a}.\text{s}.$$

Now, using conditions that h → 0 and log(n)/{nφ(h)} → 0, we see that the right hand side is in fact o_p(1) and hence the proof follows.

A.2.2 Proof of Lemma 2

Recall that under the model,

$${\mathcal{Y}}_i=X_i^{\text{T}}\beta ({\mathcal{Z}}_i)+{\epsilon }_i.$$

Define $\mathrm{\Omega }(z)=E[{\sum }_{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}]$. Then we have that

$$\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{\mathcal{Y}}_i/\mathrm{\Omega }(z)=A_1+A_2,$$

where by Ferraty and Vieu (2004) uniformly in z ∈

$$\begin{array}{l}{A}_1=\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{X}_i^{\text{T}}\beta ({\mathcal{Z}}_i)/\mathrm{\Omega }(z)\\ =E[{XX}^{\text{T}}\beta (\mathcal{Z})\mid \mathcal{Z}=z]+O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)a.s.\\ =E[{XX}^{\text{T}}\mid \mathcal{Z}=z]\beta (z)+O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)a.s.\end{array}$$ (3)

Also, using the fact that E(ε|X, ) = 0 we see that

$$\begin{array}{l}{A}_2=\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}{X}_i{\epsilon }_i/\mathrm{\Omega }(z)\\ =O(h^a)+O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)a.s.\end{array}$$

Also it is easy to see (Ferraty and Vieu, 2006) that

$$\underset{z\in C}{sup}\left|\sum _{i=1}^n{K}_h\{d(z,{\mathcal{Z}}_i)\}/\mathrm{\Omega }(z)-1\right|=O\left(\sqrt{\frac{log(n)}{n\phi (h)}}\right)=o(1)\text{a}.\text{s}.$$ (4)

Now the proof follows by combining (3) and (4).

A.3 Proof of Result 2

A.3.1 Expansion of η̂

We start by noting that

$$\begin{array}{l}0=n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}\{Y_i-S^{\text{T}}\widehat{\eta }-X_i^{\text{T}}\widehat{\beta }({\mathcal{Z}}_i,\widehat{\eta })\}\\ =n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}\{Y_i-S^{\text{T}}{\eta }_0-X_i^{\text{T}}\widehat{\beta }({\mathcal{Z}}_i,{\eta }_0)\}-n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}{(S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i)}^{\text{T}}(\widehat{\eta }-{\eta }_0).\end{array}$$

Using similar argument as in the previous section, we obtain that

$$\underset{z\in \mathcal{C}}{sup}\left|{\widehat{\beta }}_{\eta ,j}(z)-{\beta }_{\eta ,j}(z)\right|=O(h^a)+O\left(\sqrt{\frac{logn}{n\phi (h)}}\right)a.s.$$ (5)

Also it is easy to see that E(S̃X^T| ) = 0. Now we derive

$$\begin{array}{l}{n}^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}\{Y_i-S_i^{\text{T}}{\eta }_0-X_i^{\text{T}}\widehat{\beta }({\mathcal{Z}}_i,\eta )\}\\ =n^{-1}\sum _{i=1}^n\{S_i+{\beta }_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}{\epsilon }_i\\ -n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}{X}_i^{\text{T}}\{\widehat{\beta }({\mathcal{Z}}_i,{\eta }_0)-\beta ({\mathcal{Z}}_i,{\eta }_0)\}\\ +n^{-1}\sum _{i=1}^n{\{{\widehat{\beta }}_{\eta }({\mathcal{Z}}_i)-{\beta }_{\eta }({\mathcal{Z}}_i)\}}^{\text{T}}{X}_i{\epsilon }_i\\ -n^{-1}\sum _{i=1}^n{\{{\widehat{\beta }}_{\eta }({\mathcal{Z}}_i)-{\beta }_{\eta }({\mathcal{Z}}_i)\}}^{\text{T}}{X}_i{X}_i^{\text{T}}\{\widehat{\beta }({\mathcal{Z}}_i,{\eta }_0)-\beta ({\mathcal{Z}}_i,{\eta }_0)\}\\ =n^{-1}\sum _{i=1}^n{\stackrel{\sim }{S}}_i{\epsilon }_i-A_1+A_2-A_3.\end{array}$$

Using (5) and the fact that E(ε|X, ) = 0, it is obvious that A₂ = o_p(n^−1/2). Moreover, using condition (C.h), (5) and Result 1 we have A₃ = o_p(n ^−1/2). Finally, A₁ = o_p(n ^−1/2) follows from Result 1 and noting that $S_i+{\beta }_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i={\stackrel{\sim }{S}}_i$ and E(S̃X^T| ) = 0

We observe that

$$n^{-1}\sum _{i=1}^n\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}{\{S_i+{\widehat{\beta }}_{\eta }^{\text{T}}({\mathcal{Z}}_i)X_i\}}^{\text{T}}\to E(\stackrel{\sim }{S}{\stackrel{\sim }{S}}^{\text{T}})\text{a}.\text{s}.$$

Hence we have that

$$\widehat{\eta }-{\eta }_0={\{E(\stackrel{\sim }{S}{\stackrel{\sim }{S}}^T)\}}^{-1}{n}^{-1}\sum _{i=1}^n{\stackrel{\sim }{S}}_i{\epsilon }_i+o_p(n^{-1/2}),$$

and the result follows.