Parametrically guided estimation in nonparametric varying coefficient models with quasi-likelihood

Abstract

Varying coefficient models allow us to generalize standard linear regression models to incorporate complex covariate effects by modeling the regression coefficients as functions of another covariate. For nonparametric varying coefficients, we can borrow the idea of parametrically guided estimation to improve asymptotic bias. In this paper, we develop a guided estimation procedure for the nonparametric varying coefficient models. Asymptotic properties are established for the guided estimators and a method of bandwidth selection via bias-variance tradeoff is proposed. We compare the performance of the guided estimator with that of the unguided estimator via both simulation and real data examples.

1. Introduction

A common scenario in studies is when the researcher observes some covariates and a response and wants to estimate the conditional mean function of the response given the covariates. A typical approach is to fit a generalized linear model (GLM) (Nelder and Wedderburn 1972) where a parametric assumption is made on a transformation of the conditional mean. This approach is easily interpreted and efficient if the correct parametric model is chosen, but can have serious consequences when the model is misspecified, a common problem in real-world applications. Alternatively, nonparametric methods make little or no assumptions of the model and are robust to model misspecification, however, they are slower to converge (Glad 1998) and can fail when the dimensions of the covariates are too high (Fan and Zhang 2008). In this paper, we combine the benefits of both parametric and nonparametric methods by considering estimation of nonparametric varying coefficient models using a pre-specified parametric family of functions as a guide. We use a local likelihood kernel smoothing based estimation procedure and investigate the asymptotic properties of the resulting estimates. We evaluate the finite sample performance of our method via a simulation study and demonstrate it by applying to the AIDS Clinical Trials Group (ACTG) Protocol 315 data described in Section 5.

Varying coefficient models (VCMs) are nonparametric generalized additive models (Wood 2006) that increase the flexibility of linear models by using a smooth function to model the parameters. There are numerous advantages of using VCMs over their parametric linear model counterparts. First, fitting a standard linear model is too restrictive because few real world problems satisfy the assumption of linearity. The true, possibly nonlinear, relationship between the covariates are not well captured by polynomial fitting which can lead to large bias in estimation. Second, VCMs allow for the interaction between covariates to be modeled in a nonparametric way. Models with only main effects included may miss the effect from the interaction between covariates. Another advantage for using VCMs is that the dimensionality issue is avoided because the coefficient functions are allowed to vary as a function of another covariate (Hastie and Tibshirani 1993). VCMs are easy to interpret and arise when it is desirable to know how regression coefficients change over groups, or over time in longitudinal studies. These flexible models can be applied to a variety of data types, including longitudinal data (Brumback and Rice 1998; Hoover, Rice, Wu, and Yang 1998), time series data (Chen and Tsay 1993; Huang and Shen 2004), environmental data (Fan and Zhang 1999), and genetic data (Ma, Yang, Romero, and Cui 2011).

VCMs consist of smooth, functional parameters that need to be estimated and this can be done using penalized spline approaches (Eilers and Marx 1996; Cao, Lin, Wu, and Yu 2010), basis expansion methods (Holdeman 1969), or by applying regression locally (Chen and Tsay 1993; Cai, Fan, and Li 2000). In this paper, we utilize the latter method since these estimators are efficient and have nice sampling properties (Fan 1993). This method involves using a kernel function to weight the likelihood and applying polynomial regression locally using Taylor series approximations. In practice, the full likelihood may be unknown or difficult to construct, so we replace it with the quasi-likelihood introduced by Wedderburn (1974). With the quasi-likelihood, only the relationship between the mean and the variance needs to be specified, and the model will still retain most of the efficiency of a maximum likelihood estimation procedure.

Nonparametric estimators can be enhanced by using a parametric guide. In practice, previous information or exploratory analysis may give some insight on the shape of the unknown functions and this information can be used to speed up convergence and reduce bias. The basic process is as follows: 1) identify a parametric family of functions that captures the shape, 2) remove the trend and carry out local polynomial estimation, and 3) add the trend back to obtain the final estimators of the functions. This guided estimation scheme was first studied in the density estimation framework where Hjort and Glad (1995) showed that their estimator had better bias and similar variance compared to the traditional nonparametric estimator, even when the guide was superficial. It has also been studied in least squares regression (Glad 1998; Martins-Filho, Mishra, and Ullah, 2008) quasi-likelihood models (Fan, Wu, and Feng 2009) and nonparametric additive models (Fan, Maity, Wang, and Wu 2013).

In this paper, we propose a way to improve the coefficient functions in VCMs by using guided techniques. Cai et al. (2000) first proposed local estimation techniques for VCM coefficient functions using local-likelihood equations, and present asymptotic properties of their estimators. Fan et al. (2009) propose using quasi-likelihood models as a general case when the likelihood is unavailable and present guided estimation when a single covariate and response are observed. Fan et al. (2013) extend this methodology to an additive model where multiple covariates are observed but do not interact with each other. The main contribution of this paper is that we allow for interaction between two observed covariates and thus extend previous works to VCMs, of which Fan et al. (2009) is a special case. We borrow the generality of the quasi-likelihood models and the idea of parametrically guided estimation to improve the bias of our estimators, but we extend the optimal bandwidth selection methodology for the kernel weight function when we apply local polynomial regression. We also give asymptotic theory and results for the guided estimators in the VCM framework, different than the GLM and the additive model in Fan et al. (2009) and Fan et al. (2013), respectively. We show in simulations that the guided estimators have lower bias and similar variance when a fixed bandwidth is used, and lower bias and variance when the optimal bandwidth is used. We then estimate the functional parameters in the (ACTG) Protocol 315 data.

The rest of this paper is organized as follows: in Section 2 we give an overview of QLMs and the standard nonparametric estimation procedure using local polynomial fitting. We propose our parametrically guided estimation scheme in Section 2.2 using two different types of corrections, and give some asymptotic properties of our estimators. In Section 3, we present one method of choosing the bandwidth parameter in local polynomial fitting. We evaluate the performance of our estimators compared to the standard ones in Section 4 and found that our estimators had lower bias when a fixed bandwidth was used, and lower bias and variability when the optimal bandwidth was chosen. We then applied our methodology to the ACTG data in Section 5 and provide some concluding remarks in Section 6.

2. Guided Estimation for Varying Coefficient Models

Assume that for each of n subjects we observe covariates X_i = (1, X_i1, …, X_iq)^T and T_i, and a response Y_i. The VCM for these covariates is defined as

$$\begin{array}{l}g({\mu }_i)={\theta }_0(T_i)+X_{i1}{\theta }_1(T_i)+\cdots +X_{iq}{\theta }_q(T_i)\\ ={\mathbf{X}}_i^T\mathit{\theta }(T_i),\end{array}$$ (1)

where μ_i = E(Y_i|X_i, T_i) is the conditional mean of the response, g(·) is a link function from the GLM framework, and θ(·) = {θ₀(·), θ₁(·), …, θ_q (·)}^T are unknown, smooth functions. The first term models the unique effect of T and the remaining terms model the interaction between X and T. This VCM is more flexible than a linear regression model because it allows the effect of X to vary smoothly with T and the effect of T is not restricted to a linear assumption. The goal is to estimate θ(·) and there are several ways to do this (Cleveland, Grosse, and Shyu 1991; Hastie and Tibshirani 1993). The method we adopt is to use local-likelihood kernel smoothing using a quasi-likelihood.

2.1. Framework and Local Likelihood Estimation

Quasi-likelihood models (QLMs), an extension of GLMs, are ideal because often the full likelihood may be unknown or difficult to construct. In QLMs, only the relationship between the conditional mean and variance of the responses need to be specified, which is often doable in practice. The full conditional log-likelihood is replaced with a quasi-likelihood function Q(Y_i, μ_i) and if we define var(Y_i|X_i, T_i) = V (μ_i), then Q satisfies

$$\frac{\partial }{\partial \mu }Q(y,\mu )=\frac{y-\mu }{V(\mu )}.$$

Wedderburn (1974) shows that Q has similar properties to the log-likelihoods and that Q is exactly the likelihood when the response comes from a single parameter exponential family.

The standard, unguided, nonparametric procedure for estimating θ(·) is to use local polynomial fitting by first approximating the functions using a Taylor series expansion so that

$${\theta }_j(T_i)\approx {\beta }_{j0}+(T_i-t_0){\beta }_{j1}+\cdots +{(T_i-t_0)}^P{\beta }_{jP},$$ (2)

where ${\beta }_{jp}={\theta }_j^{(p)}(t_0)/p!$ for p = 0, …, P and j = 0, …, q. Substituting this approximation into (1) yields $g({\mu }_i)={\mathbf{G}}_i^T\mathit{\beta }$ where G_i = {1, (T_i − t₀)¹, …, (T_i − t₀)^P}^T ⊗ X_i, and $\mathit{\beta }={({\mathit{\beta }}_0^T,\dots ,{\mathit{\beta }}_P^T)}^T$ where β_p = (β_0p, …, β_qp)^T. The approximation in (2) is only accurate when t₀ is close to T_i, so the quasi-likelihood is weighted in such a way that more weight is given to t₀’s close to T_i and little to no weight to those far from T_i. This is done by using a kernel function K_h(·) = K(·/h)/h and defining the local quasi-likelihood as

$$\sum _{i=1}^{n}Q\{g^{-1}({\mathbf{G}}_i^T\mathit{\beta }),Y_i\}{K}_h(T_i-t_0).$$ (3)

The parameter h is the bandwidth and needs to be estimated (see Section 3). We maximize (3) with respect to β and the solution β̂₀ will be the estimate of θ(t₀).

2.2. Guided Estimation

The local likelihood estimators can be enhanced by using a parametric guide. Intuitively speaking, more curvature in a function makes it more difficult to estimate. If, through exploratory analysis or prior information, one has some idea of the shape of the true function, then one can identify a parametric family that captures this trend. Using the parametric guide, the curvature of the function can be removed yielding a flatter curve that is easier to estimate. Once this flatter curve is estimated, then the guide can be used to add the trend back and obtain the final estimate of the original function. This type of guided estimation has been shown to reduce bias of nonparametric estimators (Hjort and Glad 1995; Glad 1998; Martins-Filho et al. 2008) and improve variance since a larger bandwidth can be selected (Fan et al. 2009, 2013).

Define a parametric family that captures the trend of the function as {θ_jg(t, α_j): α_j = (α_j1, …, α_jmj)^T ∈ ⊂ ℝ^m_j} for j = 0, …, q. The optimal guides can be found by maximizing the quasi-likelihood

$$\sum _{i=1}^{n}Q[g^{-1}\{{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i,\mathit{\alpha })\},Y_i]$$

with respect to $\mathit{\alpha }={({\mathit{\alpha }}_0^T,\dots ,{\mathit{\alpha }}_q^T)}^T$ where θ_g(T_i, α) = {θ_0g(T_i, α₀), …, θ_qg(T_i, α_q)}^T. The best fit is denoted by θ̂_g(t) = θ_g(t, α̂) where we suppress the dependence on α in our notation. In this paper we present two methods of removing the trend using an additive correction or a multiplicative correction.

2.2.1. Additive Correction

If the curvature of θ_j is well approximated by θ̂_jg, then estimating the quantity θ_j(t) − θ̂_jg(t) will yield more accurate and less variable estimates since this quantity is close to flat. Once estimated, the guide is added back to give the final estimate of θ_j. This process can be achieved in one step by defining η_j(t) = θ_j(t) − θ̂_jg(t) + θ̂_jg(t₀) and estimating η_j at t₀. This definition of η_j is known as the additive correction.

Using this correction, the VCM in (1) can be rewritten as

$$g({\mu }_i)={\mathbf{X}}_i^T\mathit{\eta }(T_i)+{\mathbf{X}}_i^T\mathbf{h}(T_i)$$

where η(t) = {η₀(t), …, η_q(t)}^T, and h(t) = {θ̂_0g(t)−θ̂_0g(t₀), …, θ̂_qg(t)− θ̂_qg(t₀)}^T. Similar to Section 2.1 and with a slight abuse of notation, a Taylor series approximation is used for η_j(T_i) about the point t₀ such that

$${\eta }_j(T_i)\approx {\beta }_{j0}+(T_i-t_0){\beta }_{j1}+\cdots +{(T_i-t_0)}^P{\beta }_{jP}.$$

where ${\beta }_{jp}={\eta }_j^{(p)}(t_0)/p!$ for p = 0, …, P. The local quasi-likelihood

$$\begin{array}{l}Q(\mathit{\beta })\equiv Q(\mathit{\beta };h,t_0,\widehat{\mathit{\alpha }})\\ =\sum _{i=1}^{n}Q[g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)\},Y_i]\times K_h(T_i-t_0)\end{array}$$

is maximized with respect to β and the estimate of β̂₀ corresponds to η̂(t₀) which gives the final estimate θ̂(t₀). Because h(t) is known for fixed T = t, our model can be fit using standard software with h(t) as an offset.

2.2.2. Multiplicative Correction

An alternative correction which leads to a different guided estimator is the multiplicative correction. As in the additive case, the ratio θ_j(t)/θ̂_jg(t) will be flat if θ̂_jg(t) captures the trend of θ_j(t) and estimating this ratio will be less biased than estimating the unknown function directly. Once estimated, the ratio is then multiplied by the guide to get the final estimate of θ_j. The multiplicative correction is defined as η_j(t) = θ_j(t){θ̂_jg(t₀)/θ̂_jg(t)} and the one step solution requires estimating η_j(t) at t₀.

An alternative correction which leads to a different guided estimator is the multiplicative correction. As in the additive case, the ratio θ_j(t)/θ̂_jg(t) will be flat if θ̂_jg(t) captures the trend of θ_j(t) and estimating this ratio will be less biased than estimating the unknown function directly. Once estimated, the ratio is then multiplied by the guide to get the final estimate of θ_j. The multiplicative correction is defined as η_j(t) = θ_j(t){θ̂_jg(t₀)/θ̂_jg(t)} and the one step solution requires estimating η_j(t) at t₀.

Using the multiplicative correction, (1) is written as

$$g({\mu }_i)=\frac{{\widehat{\theta }}_{0g}(T_i)}{{\widehat{\theta }}_{0g}(t_0)}{\eta }_0(T_i)+\frac{{\widehat{\theta }}_{1g}(T_i)}{{\widehat{\theta }}_{1g}(t_0)}{X}_{i1}{\eta }_1(T_i)+\cdots +\frac{{\widehat{\theta }}_{qg}(T_i)}{{\widehat{\theta }}_{qg}(t_0)}{X}_{iq}{\eta }_q(T_i).$$

Estimating η_j(T) is achieved by first using a Taylor series expansion of η_j(t) about the point t₀ and then maximizing

$$Q(\mathit{\beta })=\sum _{i=1}^{n}Q\{g^{-1}({\mathbf{G}}_i^{\ast T}\mathit{\beta }),Y_i\}{K}_h(T_i-t_0)$$

with respect to β, where ${\mathbf{G}}_i^{\ast }=\{1,T_i-t_0,\dots ,{(T_i-t_0)}^p\}\otimes \left(\frac{{\widehat{\theta }}_{0g}(T_i)}{{\widehat{\theta }}_{0g}(t_0)},X_{i1}\frac{{\widehat{\theta }}_{1g}(T_i)}{{\widehat{\theta }}_{1g}(t_0)},\dots ,X_{iq}\frac{{\widehat{\theta }}_{qg}(T_i)}{{\widehat{\theta }}_{qg}(t_0)}\right)$, and ⊗ denotes the Kronecker product between the two vectors. The solution β̂₀ give our final estimates of θ̂(t₀). By manipulating the design matrix, there is no offset for the multiplicative correction and this model can easily be fit using standard GLM software.

2.3. Asymptotic Properties

In this section, we investigate the asymptotic properties of the proposed guided estimators. For illustration purposes, we present the case where estimation is performed using a local linear approximation (P = 1) and an additive guide. Similar results for general P and the multiplicative guide can be obtained by straightforward but tedious algebra.

Define κ_d = ∫u^dK(u) du and ν_d = ∫u^dK²(u) du. Define M to be a 2 × 2 matrix with elements M_kl = κ_k+l−2 and R to be a 2 × 2 matrix with elements R_kl = ν_k+l−2. Let ρ(x, t) = 1/[V {μ(x, t)}g′²{μ(x, t)}] and γ(x, t₀) = var(Y₁|X₁ = x, T₁ = t₀)/[V{μ(x, t₀)}g′{μ(x, t₀)}]². Make the definitions

$$\begin{array}{l}{\mathbf{V}}_1(t_0)=f_t(t_0)\mathbf{M}\otimes E\{\rho ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_1=t_0\},\\ {\mathbf{V}}_2(t_0)=f_t(t_0)\mathbf{R}\otimes E\{\gamma ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_1=t_0\},\\ {\mathbf{B}}_1(t_0)={\mathbf{M}}^{-1}{({\kappa }_2,{\kappa }_3)}^T\otimes {\mathit{\eta }}^{″}(t_0)/2,\end{array}$$

where f_t(·) is the marginal density function of T. Then for a fixed guide, we have the following result.

Theorem 2.1

Fix a point t₀ and assume the guide is fixed. Under the conditions stated in Section A.1 of the Appendix, as h → 0, nh → ∞, and nh⁵ → constant, we have

$${(nh)}^{1/2}\{\widehat{\mathit{\theta }}(t_0)-\mathit{\theta }(t_0)-h^2{\mathit{B}}_1(t_0)\}{\to }^d\mathrm{N}(\mathit{0},\mathbf{\sum }),$$

where Σ is the leading (q + 1) × (q + 1) submatrix of ${\mathit{V}}_1^{-1}{\mathit{V}}_2{\mathit{V}}_1^{-1}$.

There are two noteworthy points to make about Theorem 1. The first is that if there is no (or constant) guide and the model belongs to a one-parameter exponential family with the canonical link and correctly specified variance function, then η″(t₀) = θ″(t₀), ρ(x, t) = γ(x, t) = V {μ(x, t)}, and our result reduces to Theorem 1 of Cai et al. (2000). The second is that only the bias term B₁(t₀) is affected when a guide is used, and not the asymptotic variance. To be specific, consider the integrated squared bias for each function θ_j(·). It is evident that the integrated squared bias of the guided estimate is smaller that that of the unguided one when $\int {\{{\theta }_j^{″}(t)-{\theta }_{jg}^{″}(t)\}}^{2}dt<\int {\{{\theta }_j^{″}(t)\}}^{2}dt$. This is analogous to Remark 3 of Fan et al. (2009). Therefore when a fixed bandwidth is used, the squared bias (and hence the MSE) is reduced if an appropriate guide is chosen. Furthermore, finding the optimal bandwidth using the procedure from Section 3 allows for a larger bandwidth to be selected compared to the unguided estimate which will reduce the variance as well as the bias. We demonstrate this in our simulation study in Section 4.

Theorem 2.1 assumes a fixed guide but in practice the guide needs to be estimated. Theorem 2.2 states that the expressions above are still valid for estimated guides as well.

Theorem 2.2

Let f_jnt(x, t, y) = f_t(t) exp(Q[g⁻¹{x^T θ(t)}, y]) be the true joint density and f_prp(x, t, y, α) = f_t(t) exp(Q[g⁻¹{x^Tθ_g(t, α)}, y]) be the proposed joint density. Define α^* to be the minimizer of the Kullback-Leibler distance between f_prp and f_jnt. Then under the White (1982)-type conditions in the Appendix, the same result as in Theorem 2.1 holds when an estimated guide θ̂_g(·) is used in place of a fixed guide with the modification that α is now replaced by α^*.

The proofs of Theorems 2.1 and 2.2 are given in the Appendix.

Remark 1

The asymptotic results presented in this section are an extension of the results from Fan et al. (2009). Consider the special case where the covariate contains only the intercept X_i = 1 and parameter function θ(·) ≡ θ₀(·). Then we have ρ(x, t) = ρ(t) = 1/[V {μ(t)}g′²{μ(t)}], γ(x, t) = γ(t) = var(Y₁|T₁ = t)/[V {μ(t)}g′{μ(t)}]², V₁(t₀) = f_t(t₀)ρ(t₀)M, V₂(t₀) = f_t(t₀)γ(t₀)R, and B₁(t₀) = M⁻¹(κ₂, κ₃)^T η″(t₀)/2. With these simplified definitions, the result in Theorem 2.1 implies that

$${(nh)}^{1/2}\{\widehat{\theta }(t_0)-\theta (t_0)-h^2{\kappa }_0{\kappa }_2{\eta }^{″}(t_0)/2\}{\to }^d\mathrm{N}(0,\sigma (t_0)),$$

where σ(t₀) = ν₀var(Y₁|T₁ = t₀)g′²{μ(t)}/f_t (t₀). This result is as in Theorem 1 in Fan et al. (2009) with the assumption that κ₀ = ∫K(z) dz = 1.

3. Optimal Bandwidth Selection

Note that for simplicity of presentation, we will consider our bandwidth selection method using the additive correction; the multiplicative correction follows easily by using the multiplicative definition of η_j(·), replacing G_i with ${\mathbf{G}}_i^{\ast }$, and omitting the offset term ${\mathbf{X}}_i^T\mathbf{h}(T_i)$.

Once β̂ is obtained, the bias arises from the approximation term of the Taylor series expansions. Hence, using more terms in the series should theoretically produce less bias. Let $r_j(T_i)={\eta }_j(T_i)-{\sum }_{k=0}^P{\eta }_j^{(k)}(t_0){(T_i-t_0)}^k/k!$ be the approximation error. If a higher order Taylor approximation is substituted for η_j(T_i) in r_j, then $r_j(T_i)\approx {\sum }_{k=1}^a{\eta }_j^{(p+k)}(t_0){(T_i-t_0)}^{p+k}/(p+k)!\stackrel{\text{def}}{=}{r}_{ji}$. We then maximize the local quasi-likelihood including the approximation errors

$$Q^{\ast }(\mathit{\beta })=\sum _{i=1}^{n}Q[g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)+{\mathbf{X}}_i^T{\mathbf{r}}_i\},Y_i]\times K_h(T_i-t_0)$$

with respect to β where r_i = (r_0i, …, r_qi)^T. Define the maximizer as β̂^*. The local quasi-likelihood Q^*(β) is differentiated with respect to β to get the gradient vector

$${Q^{\ast }}^{\prime }(\mathit{\beta })=\sum _{i=1}^n\frac{Y_i-g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)+{\mathbf{X}}_i^T{\mathbf{r}}_i\}}{V[g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)+{\mathbf{X}}_i^T{\mathbf{r}}_i\}]}{(g^{-1})}^{\prime }({\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)+{\mathbf{X}}_i^T{\mathbf{r}}_i){\mathbf{G}}_i{K}_h(T_i-t_0)$$

and the second derivative is taken to get the Hessian matrix Q^*″(β). A Taylor series expansion is then applied to Q^*′ around β̂ to get

$${Q^{\ast }}^{\prime }({\widehat{\mathit{\beta }}}^{\ast })\approx {Q^{\ast }}^{\prime }(\widehat{\mathit{\beta }})+{Q^{\ast }}^{″}(\widehat{\mathit{\beta }})({\widehat{\mathit{\beta }}}^{\ast }-\widehat{\mathit{\beta }})=\mathbf{0}$$

and thus, an approximation of the estimation bias is β̂ − β̂^* ≈ {Q^*″(β̂)}⁻¹Q^*′(β̂).

To get an approximation of the variance, a Taylor series expansion of Q′(β̂) is done about the true β, denoted by β⁰. Note that

$$\mathbf{0}=Q^{\prime }({\widehat{\mathit{\beta }}}^{\ast })\approx Q^{\prime }({\mathit{\beta }}^0)+Q^{″}({\mathit{\beta }}^0)(\widehat{\mathit{\beta }}-{\mathit{\beta }}^0)$$

which implies β̂ − β⁰ ≈ −{Q″(β⁰)}⁻¹Q′(β⁰), and the estimate of the conditional variance is

$$var(\widehat{\mathit{\beta }}-{\mathit{\beta }}^0\mid \mathbb{X},\mathbb{T})\approx {\{Q^{″}({\mathit{\beta }}^0)\}}^{-1}var\{Q^{\prime }({\mathit{\beta }}^0)\mid \mathbb{X},\mathbb{T}\}{\{Q^{″}({\mathit{\beta }}^0)\}}^{-1}$$

where Q″(β⁰) can be approximated by Q″(β̂). To approximate the variance term, note that

$$\begin{array}{l}var\{Q^{\prime }({\mathit{\beta }}^0)\mid \mathbb{X},\mathbb{T}\}=\sum _{i=1}^{n}var\left(\frac{\partial }{\partial \mathit{\beta }}Q[g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)\},Y_i]\mid {\mathbf{X}}_i,T_i\right)K_h^2(T_i-t_0)\\ =\sum _{i=1}^{n}var\left(\frac{Y_i-g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)\}}{V[g^{-1}\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)\}]}{(g^{-1})}^{\prime }\{{\mathbf{G}}_i^T\mathit{\beta }+{\mathbf{X}}_i^T\mathbf{h}(T_i)\}\mid {\mathbf{X}}_i,T_i\right)\times {\mathbf{G}}_i{\mathbf{G}}_i^T{K}_h^2(T_i-t_0)\\ \approx \frac{{[{(g^{-1})}^{\prime }\{{\mathbf{X}}_i^T\mathit{\theta }(t_0)\}]}^2}{V[g^{-1}\{{\mathbf{X}}_i^T\mathit{\theta }(t_0)\}]}{\mathbf{G}}_i{\mathbf{G}}_i^T{K}_h^2(T_i-t_0).\end{array}$$

The last approximation follows from the fact that T_i only has significant weight in the neighborhood of t₀.

To compute the MSE, we denote the bias of θ̂ as B(t₀; h) = {B₀(t₀; h), …, B_q (t₀; h)}^T corresponding to the first q+1 components of [Q^*″(β̂)]⁻¹Q^*′(β̂). The variance-covariance matrix V(t₀; h) of θ̂ is the first (q + 1) × (q + 1) submatrix of var(β̂ − β⁰|x, t) and thevvariance of the estimated VCM ${\mathbf{X}}_i^T\widehat{\mathit{\theta }}$ is ${\mathbf{X}}_i^T\mathbf{V}(t_0;h){\mathbf{X}}_i$. The conditional MSE of X^T θ̂ given X = x is

$$\text{MSE}(t_0;h)={\mathbf{x}}^T\{\mathbf{B}(t_0;h)\mathbf{B}{(t_0;h)}^T+\mathbf{V}(t_0;h)\}\mathbf{x}.$$

The sample MSE is derived as

$$\begin{array}{l}\widehat{\text{MSE}}(t_0;h)=\frac{1}{h}\sum _{i=1}^n{\mathbf{X}}_i^T\{\mathbf{B}(t_0;h)\mathbf{B}{(t_0;h)}^T+\mathbf{V}(t_0;h)\}{\mathbf{X}}_i\\ =\text{trace}\left[\{\mathbf{B}(t_0;h)\mathbf{B}{(t_0;h)}^T+\mathbf{V}(t_0;h)\}{n}^{-1}\sum _{i=1}^n{\mathbf{X}}_i{\mathbf{X}}_i^T\right].\end{array}$$

We propose to choose h such that

$$\widehat{h}={\text{argmin}}_h\int \widehat{\text{MSE}}(t;h)dt.$$

To summarize, we use a grid of t₀’s and a grid of candidate bandwidths. We fit the local quasi-likelihood for each t₀ and h candidate and calculate the extended residual squares criterion (ERSC) defined as ERSC(x, t; h) = σ̂²{1+(p+1)/N}, where σ̂² is the weighted residual sum of squares after fitting a local pth-order polynomial and N is the number of local data points (see (5.6) of Fan, Farmen, and Gijbels (1998) for details). We then sum the ERSCs over t₀ and the bandwidth with the lowest sum becomes the pilot bandwidth. Using this bandwidth, we fit a local quasi-likelihood for each t₀ to obtain β̂. For a new set of candidate bandwidths, we fit the local quasi-likelihood using higher order Taylor series approximation and obtain β̂^*, which is theoretically more accurate. We compute the bias and variance using the gradient and Hessian of the quasi-likelihood, compute the MSE and the candidate bandwidth with the lowest MSE is our optimal bandwidth.

4. Simulation

We conducted a simulation study to evaluate the performance of our estimators. We generated each observation (X_i, T_i, Y_i) by first simulating the covariates X_i and T_i from a uniform distribution. Then, given X_i and T_i, the conditional mean μ_i of the response was generated as

$${\mu }_i=g^{-1}\{{\theta }_0(T_i)+X_{i1}{\theta }_1(T_i)\}$$

where g is the canonical link. We used a grid of equally spaced values t_k for k = 1, …, K = 100 to estimate the two functions. A cubic guide θ_0g(t, α₀) = α₀₁ + α₀₂t + α₀₃t² + α₀₄t³ was used for estimating θ₀ and a quadratic guide θ_1g(t, α₁) = α₁₁ + α₁₂t + α₁₃t² was used for estimating θ₁. We used local linear polynomial estimators with the Epanechnikov kernel weight, and for bias calculations, we chose the degree of the Taylor expansion to be a = 1.

For R = 1000 simulations, we generated the data and estimated the parameters for the cubic and quadratic guides as described above. To get the final estimates θ̂₀ and θ̂₁, we maximized the local quasi-likelihood with the appropriate distribution and canonical link, and Epanechnikov kernel weight. The design matrix of the local likelihood was constructed using both the additive and multiplicative corrections (G_i and ${\mathbf{G}}_i^{\ast }$, respectively). To find the optimal smoothing bandwidth, we first simulated 15 data sets and applied our methods from Section 3. We took the median of these 15 values as the optimal bandwidth and used that value as fixed for the 1000 simulations. We also compared the two methods using the optimal bandwidth from the unguided method as the fixed bandwidth.

Once the two functions were estimated, we computed the marginal squared bias, marginal variance, and marginal MSE of each. Define $B_{jk}=R^{-1}{\sum }_{r=1}^R[{\widehat{\theta }}_{j,r}(t_k)-{\theta }_j(t_k)],V_{jk}=R^{-1}{\sum }_{r=1}^R{[{\widehat{\theta }}_{j,r}(t_k)-R^{-1}{\sum }_{r^{\prime }=1}^R{\widehat{\theta }}_{j,r^{\prime }}(t_k)]}^2$, and ${\text{MSE}}_{jk}=B_{jk}^2+V_{jk}$ where j = 0, 1 and r indexes the simulation. The average marginal squared bias of θ̂_j is $B_j^2=K^{-1}{\sum }_{k=1}^K{B}_{1k}^2$, the average marginal variance is $V_j=K^{-1}{\sum }_{k=1}^K{V}_{jk}$ and the average marginal MSE is ${\text{MSE}}_j=K^{-1}{\sum }_{k=1}^K{\text{MSE}}_{jk}$. However instead of averaging over all k, we used the 10% trimmed mean. Tables A1 – A3 show the results of our simulations. “Best h” is each estimation method’s optimal smoothing bandwidth obtained via the bias-variance tradeoff in Section 3. All values for squared bias, variance, and MSE in the table are multiplied by 100. “Same h” refers to the fixed bandwidth obtain from the optimal bandwidth from the unguided estimators.

Table A1.

Results of trimmed average bias, variance, and MSE for Example 1. “Same h” refers to fixed bandwidth and best “h” refers to optimal bandwidth. All values are multiplied by 100.

			Bias2			Variance			MSE
			Naive	Additive	Multiplicative	Naive	Additive	Multiplicative	Naive	Additive	Multiplicative
n = 100	Same h	θ0(t)	0.129	0.022	0.022	0.179	0.173	0.174	0.308	0.195	0.196
		θ1(t)	0.003	0.001	0.001	0.552	0.522	0.522	0.556	0.522	0.522
	Best h	θ0(t)	0.129	0.098	0.096	0.179	0.104	0.106	0.308	0.203	0.202
		θ1(t)	0.003	0.001	0.001	0.552	0.294	0.296	0.556	0.295	0.297
n = 200	Same h	θ0(t)	0.080	0.014	0.015	0.090	0.088	0.089	0.171	0.103	0.104
		θ1(t)	0.003	0.001	0.001	0.269	0.261	0.262	0.272	0.262	0.262
	Best h	θ0(t)	0.080	0.026	0.026	0.090	0.074	0.075	0.171	0.100	0.101
		θ1(t)	0.003	0.001	0.001	0.269	0.217	0.217	0.272	0.217	0.217

Table A3.

Results of trimmed average bias, variance, and MSE for Example 3. All values are multiplied by 100.

			Bias2		Variance		MSE
			Naive	Additive	Naive	Additive	Naive	Additive
n=500	Same h	θ0(t)	0.84	0.17	62.43	67.23	63.27	67.40
		θ1(t)	0.21	0.02	27.10	29.08	27.31	29.10
	Best h	θ0(t)	0.84	0.18	62.34	57.60	63.27	57.77
		θ1(t)	0.21	0.02	27.10	24.95	27.31	24.96

Example 1: Poisson Response

For the Poisson response, n = 100 or 200 covariates X_i and T_i were generated with X_i ~ Unif[−1, 1] and T_i ~ Unif[−2, 2]. The true functions were θ₀(t) = sin(πt/2) + 4 and θ₁(t) = sin(πt/4 − π/2)/2 + 1. The response Y_i was generated from a Poisson(μ_i). We used a grid of K = 100 equally spaced values in [−2, 2] for t₀ to estimate the two functions. Table A1 gives the (trimmed) average marginal squared bias, variance, and MSE for the two functions estimated by the original method using no guide and by our method using guided estimation with additive and multiplicative corrections. When the same bandwidth is used the guided estimation procedure reduces bias but has no effect on variance. When the optimal bandwidth is used, the guided estimates have lower bias and lower variance. This is because the guides account for much of the trend in the true curve and the nonparametric correction is flatter and easier to estimate, resulting in lower bias. As the sample size increased, we saw a reduction in bias, variance and MSE.

Example 2: Normal Response

For the Gaussian response, n = 100 or 200 covariates X_i and T_i were generated with X_i ~ Unif[−1, 1] and T_i ~ Unif[−2, 2]. The true functions were θ₀(t) = sin(πt/2) − 2 and θ₁(t) = 2 sin(πt/4 − π/2) + 3. The response Y_i was generated from a Normal(μ_i, 1). Table A2 gives the (trimmed) average marginal squared bias, variance, and MSE for the two functions estimated by the original method using no guide and by our method using guided estimation with additive and multiplicative corrections. In this example, the guided estimates still have lower bias and variance when the optimal bandwidth is used. When the same bandwidth is used, the variance of the additive and multiplicative correction is slightly higher than the unguided estimates. The gains in bias reduction are counteracted by the higher variance so the MSE is approximately the same for both methods.

Table A2.

Results of trimmed average bias, variance, and MSE for Example 2. All values are multiplied by 100.

			Bias2			Variance			MSE
			Naive	Additive	Multiplicative	Naive	Additive	Multiplicative	Naive	Additive	Multiplicative
n = 100	Same h	θ0(t)	1.05	0.11	0.10	3.82	4.15	4.29	4.86	4.26	4.39
		θ1(t)	0.20	0.02	0.02	11.68	11.81	12.21	11.89	11.83	12.23
	Best h	θ0(t)	1.05	0.22	0.21	3.82	3.41	3.51	4.86	3.63	3.73
		θ1(t)	0.20	0.04	0.03	11.68	8.73	9.92	11.89	8.77	8.96
n = 200	Same h	θ0(t)	0.46	0.07	0.07	2.20	2.31	2.38	2.66	2.38	2.45
		θ1(t)	0.13	0.01	0.01	6.65	6.70	6.85	6.78	6.71	6.85
	Best h	θ0(t)	0.46	0.11	0.11	2.20	2.00	2.05	2.66	2.11	2.16
		θ1(t)	0.13	0.01	0.01	6.65	5.55	5.63	6.78	5.56	5.64

Example 3: Bernoulli Response

For the Bernoulli response, we chose a larger sample size of n = 500 since the estimation of the success probability is more difficult than estimating the mean in the Gaussian and Poisson case. The covariates X_i and T_i were generated with X_i ~ Unif[1, 2] and T_i ~ Unif[-1, 1]. The true functions were θ₀(t) = sin(πt)/2+1 and θ₁(t) = 0.7 sin{(t+1)π/2}−1. The response Y_i was generated from a Bernoulli(μ_i). The results from this example are presented in Table A3. Using a multiplicative correction with Bernoulli data is very unstable due to the possibility of dividing by zero, thus this correction was not used. We see that using the guides reduces bias and variance when the optimal bandwidth is used, and reduces bias but has little effect on variance when a fixed bandwidth is used.

5. HIV Data Analysis

In the AIDS Clinical Trials Group (ACTG) Protocol 315, 48 individuals infected with HIV-1 were given potent antiviral medicine to evaluate the efficacy of treatment on the reduction of viral load (plasma HIV-1 RNA). The viral load was measured repeatedly over three months and 31 baseline covariates were measured for each individual. Details of this study can be found in Lederman et al. (1998) and Liang, Wu, and Carroll (2003). Of these 31 covariates, Wu and Wu (2002) identified those that were significant predictors and in this illustration, we chose two of these covariates that appeared frequently in their models, baseline viral load and baseline CD4+ counts. Viral load is the number of copies per milliliter and is measured on a logarithmic scale with base 10. CD4+ counts are the number of lymphocytes that are CD4+. The response was the change from baseline viral load measured at day 7. If an individual did not have a viral load measurement on day 7, then the preceding or following day measurement was used. The data for our illustration are presented in Figure A1. We would like to determine if baseline viral load and baseline CD4+ counts have an effect on the change from baseline viral load measurement while adjusting for interaction between them.

Figure A1.

Scatterplot of the HIV Data variables of interest presented in Section 5. The response is change from baseline viral load at day 7.

Using a grid of size of 25 for t₀, the Epanechnikov kernel, and the Gaussian log-likelihood for Q, we fit the model in (1) and estimated θ₀ and θ₁ using the original unguided nonparametric method, represented by the solid red line in Figure A2. The pre-asymptotic bandwidth selector gave bandwidth 0.67. Based on the shape of these unguided estimates, we chose a cubic guide for our guided estimation of θ₀ and θ₁ and used the additive correction. The results from our fit are represented by the green dot-dashed lines in Figure A2 and the guides used are the black dashed line. The bandwidth for our method was also 0.67. Our estimated θ̂₀ had more curvature than the naive counterpart and followed the parametric guide very closely, suggesting that there is little model misspecification when using a cubic guide for θ₀. For θ̂₁, the naive estimate and our guided estimate had somewhat similar shapes, with the most difference in the left endpoints.

Figure A2.

Nonparametric estimate (solid red) and parametrically guided estimate (green dot-dashed) of θ₀ (a) and θ₁ (b) along with the cubic guide (black dashed).

The point-wise bootstrap 95% confidence intervals are given in Figure A3. The confidence intervals for the guided estimates were slightly wider in the boundaries but overall were similar to those of the naive estimate. In Figure A3(c) and (d), the entire confidence interval for the functions contain zero. Recall that θ₁ is the slope function and this term models the interaction between the two covariates. This indicates that there is no interaction between baseline viral load and baseline CD4+ counts and the response can be adequately modeled as a cubic function of baseline viral load alone.

Figure A3.

Nonparametric estimate (solid red) of θ₀ (a) and θ₁ (c) and parametrically guided estimate (solid green) of θ₁ (b) and θ₂ (d) along with a point-wise 95% bootstrap confidence intervals (black dashed).

We also used the VCM to separately model the viral load after day 14, day 21, and day 28 with the same two covariates in order to compare the estimated functions on different days to day 7. Again, if an individual did not have a viral load measurement on these exact days, then the preceding or following day was used, and if all three days were missing, the individual was dropped from the analysis. This yielded 35 individuals for the day 14 analysis, 39 individuals for the day 21 analysis, and 38 individuals for the day 28 analysis. The estimated functions corresponding to these responses are given in Figure A4. The pre-asymptotic bandwidth selector gave bandwidths 0.53, 1.22 and 0.77 for the naive estimates of day 14, 21, and 28 respectively. The bandwidth for the guided estimates were 0.63, 1.26, and 0.77 for day 14, 21, and 28 respectively. The shape of the slope function θ₁ changes drastically for the different days, but the entire confidence interval for all three θ₁ functions (not presented) contains zero. Thus CD4+ has a very different interaction effect on baseline viral load for different days, but the overall effect is not significant. The shape of the intercept function θ₀ was similar for days 14 and 21, and had more of a cubic shape for day 28. Similar to day 7, the parametrically guided estimates of θ₀ followed their respective guides very closely.

Figure A4.

Nonparametric estimate (solid red) and parametrically guided estimate (green dot-dashed) of θ₀ (a)–(c) and θ₁ (d)–(f) for day 14, day 21, and day 28 responses. A cubic guide was used in (a), (c), and (f), a quadratic guide was used in (b) and (e), and a quartic guide was in (d).

6. Discussion

In this paper, we used parametric guides to enhance the performance of nonparametric estimators of the parameter functions in varying coefficient models. We generalized to quasi-likelihoods since the true likelihood is often unavailable. We presented two ways of using the guide and estimated the corrected functions using local polynomial fitting. We developed the asymptotic properties of the guided estimators and a method of selecting the optimal bandwidth parameter of the kernel function. We conducted a simulation study to compare our guided estimators to their standard nonparametric equivalents, and found that the guided estimators had lower bias when a fixed bandwidth was used, and lower bias and variance when the optimal bandwidth was used. In general, even if the shape of the parameter function is not captured by the guide, the guided estimator will still have better bias than the unguided counterpart and the two will have similar variability.

In this paper we present the additive and multiplicative correction which are special cases of a unified family of guided estimators proposed by Fan et al. (2009). This work could be extended to include this unified family and its asymptotic properties. Other future work includes extending our methods to functional data where the covariates of interest are smooth functions and the response can be functional or scalar. The functional covariates correspond to unknown functional parameters that need to be estimated, and this can be done using our guided estimation scheme. This work can also be extended to multivariate unknown functions (e.g. θ (T_i)) by using an empirical basis expansion of the function and reducing it to a VCM.

Appendix A. Appendix

A.1. Assumptions, Definitions and Facts

Recall that X_i = (1, X_i1, …, X_iq)^T and that G_i = {1, (T_i − t₀)}^T ⊗ X_i for P = 1. Also recall that ρ(x, t) = 1/[V{μ(x, t)}g′²{μ(x, t)}]. Define Q_k(u, v) = ∂^kQ{g⁻¹(u), v}/∂u^k. Since Q(·) is a quasi-likelihood function, Q_k(u, v) is linear in v for each fixed u. Thus for fixed X = x, we have

$$\begin{array}{l}{Q}_1\{{\mathbf{x}}^T\mathit{\theta }(t_0),\mu (\mathbf{x},t_0)\}=0,\\ Q_2\{{\mathbf{x}}^T\mathit{\theta }(t_0),\mu (\mathbf{x},t_0)\}=-\rho (\mathbf{x},t_0).\end{array}$$

The following assumptions that we use to prove Theorem 2.1 and 2.2 are well used in the nonparametric regression and nonparametric varying coefficient modeling literature, see for examples Cai et al. (2000) and Fan et al. (2009).

• The kernel K(·) is a symmetric positive bounded function with [−1, 1] as support, and follows the property that ∫ uK(u) du = 0.

• E(|X|³|T = t₀) is continuous at t₀.

• E(Y⁴|X = x, T = t₀) is bounded in a neighborhood of t₀.

• The function Q₂(u, v) < 0 for each u and v in the range of the response variable.

• The functions f_t(·), V₁(·), V{μ(x, ·)}, V′{μ(x, ·)} and g‴{μ(x, ·)} are continuous at t₀. Also, f_t(t₀) > 0 and V₁(t₀) > 0.

• The functions ${\theta }_j^{″}(·)$ for j = 0, …, q are continuous in a neighborhood around t₀.

For our results to be valid for the case when the guide is estimated, we use the following White (1982)-type condition, see e.g., Fan et al. (2009). Recall that we denote the guides as θ_jg(t) = θ_jg(t, α) and similarly θ̂_jg(t) = θ_jg(t, α̂), omitting the dependence on α to ease notation.

• We assume that E[log{f_jnt(x, t, y)}] exists. Also, there exists a function m(x, t, y) such that |f_prp(x, t, y)}|≤m(x, t, y).

• E[log{f_jnt(x, t, y)} – log{f_prp(x, t, y)}] has a unique minimizer α^*.

• Each element (∂/∂α) log{f_prp(x, t, y)} is continuously differentiable in α.

• There are functions m₂(x, t, y) and m₃(x, t, y) such that for any α, the absolute value of each element of [(∂/∂α) log{f_prp(x, t, y)}][(∂/∂α) log{f_prp(x, t, y)}]^T is bounded by m₂(x, t, y), and that the absolute value of each element of [(∂²/∂αα^T) log{f_prp(x, t, y)}] is bounded by m₃(x, t, y). Also E{m₂(x, t, y)} and E{m₃(x, t, y)} exist.

• The matrix E [(∂/∂α) log{f_prp(x, t, y)}][(∂/∂α) log{f_prp(x, t, y)}]^T is nonsingular at α^* and that α^* is a regular point of the matrix E[(∂²/∂αα^T) log{f_prp(x, t, y)}].

A.2. Proof of Theorem 2.1

We first prove the result with a fixed guide and then address the case when the guide is estimated.

Fix a point t₀. Recall that η_j(t) = θ_j(t) − θ_jg(t) + θ_jg(t₀). Define b_n = (nh)^−1/2 and $\mathit{\delta }=b_n^{-1}{[{\beta }_{00}-{\theta }_0(t_0),\dots ,{\beta }_{q0}-{\theta }_q(t_0),h\{{\beta }_{01}-{\eta }_0^{\prime }(t_0)\},\dots ,h\{{\beta }_{q1}-{\eta }_q^{\prime }(t_0)\}]}^T$. Then it is straightforward to see that δ̂ minimizes

$$\sum _{i=1}^{n}Q[g^{-1}\{b_n{\mathbf{G}}_i^T\mathit{\delta }+s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\}$$

as a function of δ, where $s(T_i,t_0)={\mathbf{X}}_i^T\{\mathit{\eta }(t_0)+(T_i-t_0){\mathit{\eta }}^{\prime }(t_0)\}$. By Taylor’s expansion we obtain

$$\begin{array}{l}\mathcal{L}(\mathit{\delta })\equiv \sum _{i=1}^{n}Q[g^{-1}\{b_n{\mathbf{G}}_i^T\mathit{\delta }+s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\}\\ ={\mathcal{L}}_0+{\mathcal{L}}_1^T\mathit{\delta }+{\mathit{\delta }}^T{\mathcal{L}}_2\mathit{\delta }/2+{\mathcal{L}}_3(\mathit{\delta }),\end{array}$$

where

$$\begin{array}{ll}\hfill {\mathcal{L}}_0& =\sum _{i=1}^{n}Q[g^{-1}\{s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\};\hfill \\ \hfill {\mathcal{L}}_1& =b_n\sum _{i=1}^n{Q}_1[\{s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\}{\mathbf{G}}_i;\hfill \\ \hfill {\mathcal{L}}_2& =b_n^2\sum _{i=1}^n{Q}_2[\{s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\}{\mathbf{G}}_i{\mathbf{G}}_i^T;\hfill \\ \hfill {\mathcal{L}}_3(\mathit{\delta })& =(b_n^3/6)\sum _{i=1}^n{Q}_3[\{s_i^{\ast }+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},Y_i]K\{(T_i-t_0)/h\}{({\mathbf{G}}_i^T\mathit{\delta })}^3\hfill \end{array}$$

with $s_i^{\ast }$ lying between s(T_i, t₀) and $b_n{\mathbf{G}}_i^T\mathit{\delta }+s(T_i,t_0)$.

We analyze each term separately. First we note that using similar argument as in Cai et al. (2000) and Fan et al. (2009), E(| (δ)|) is bounded by a term of the order

$$O({nb}_n^{3}E\mid Q_3(s_i^{\ast },Y_1){({\mathbf{X}}_1^T\mathit{\delta })}^{3}K\{(T_i-t_0)/h\}\mid )=O(b_n).$$ (A1)

The previous line makes use of the assumption that K(·) is bounded, Q₃(u, v) is linear in v, and that E(|X|³|T = t₀) is continuous at t₀.

Next, we take up . For each T_i such that |T_i − t₀| < h, a Taylor’s series expansion gives us

$${\mathbf{X}}_i^T\mathit{\eta }(T_i)=s(T_i,t_0)+{(T_i-t_0)}^2{\mathbf{X}}_i^T{\mathit{\eta }}^{″}(t_0)/2+o_p(h^2).$$ (A2)

Thus we have

$$Q_2[\{s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(t_i)\},\mu ({\mathbf{X}}_i,T_i)]=Q_2[{\mathbf{X}}_i^T\mathit{\theta }(T_i),\mu ({\mathbf{X}}_i,T_i)]+o_p(1)=-\rho ({\mathbf{X}}_i,T_i)+o_p(1).$$

The expected value of the (j, k)th element of , with 1 ≤ j, k ≤ (q + 1) is

$$\begin{array}{l}E({\mathcal{L}}_{2,jk})=h^{-1}\int Q_2[\{s(T_1,t_0)+{\mathbf{X}}_1^T{\mathit{\theta }}_g(T_1)\},\mu ({\mathbf{X}}_1,T_1)]X_{1j}{X}_{1k}\times K\{(T_1-t_0)/h\}{f}_{\mathbf{x},t}({\mathbf{X}}_1,T_1){dT}_{1}d{\mathbf{X}}_1\\ \int Q_2[\{s(wh+t_0,t_0)+{\mathbf{X}}_1^T{\mathit{\theta }}_g(wh+t_0)\},\mu ({\mathbf{X}}_1,wh+t_0)]X_{1j}{X}_{1k}\times K(w)f_{\mathbf{x},t}({\mathbf{X}}_1,wh+t_0)dwd{\mathbf{X}}_1\\ =-\int \rho ({\mathbf{X}}_1,wh+t_0)X_{1j}{X}_{1k}K(w)f_{\mathbf{x},t}({\mathbf{X}}_1,wh+t_0)dwd{\mathbf{X}}_1+o_p(1)\\ =-f_t(t_0)E[\rho ({\mathbf{X}}_1,T_1)X_{1j}{X}_{1k}\mid T_1=t_0]+o_p(1).\end{array}$$

Similarly, it is fairly straightforward to derive that

$$E({\mathcal{L}}_2)=-f_t(t_0)\mathbf{M}\otimes E[\rho ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_1=t_0]+o(1),$$

where M is a 2 × 2 matrix with elements M_kl = κ_k+l-2. Using similar calculations, we can show that var( ) = O{(nh)⁻¹}. Thus we have that

$${\mathcal{L}}_2=-f_t(t_0)\mathbf{M}\otimes E[\rho ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_i=t_0]+o_p(1)=-{\mathbf{V}}_i(t_0)+o_p(1).$$ (A3)

Hence we can write

$$\mathcal{L}(\mathit{\delta })-{\mathcal{L}}_0={\mathcal{L}}_1^T\mathit{\delta }-{\mathit{\delta }}^T{\mathbf{V}}_1(t_0)\mathit{\delta }+o_p(1).$$

Note that δ̂ actually minimizes − since is free of d. Now we use the quadratic approximation lemma (Fan and Gijbels 1996) and derive that

$$\widehat{\mathit{\delta }}={\mathbf{V}}_1^{-1}{\mathcal{L}}_1+o_p(1),$$

provided that is a sequence of stochastically bounded vectors.

To prove the asymptotic normality of δ̂, it is sufficient to show the same for . Note that using (A2) we have

$$Q_1[\{s(T_i,t_0)+{\mathbf{X}}_i^T{\mathit{\theta }}_g(T_i)\},\mu ({\mathbf{X}}_i,T_i)]=\rho ({\mathbf{X}}_i,T_i){(T_i-t_0)}^2{\mathbf{X}}_i^T{\mathit{\eta }}^{″}(t_0)/2+o_p(h^2).$$

Denote the joint density of X and T by f_x,t(·, ·). Using the assumption that nh⁵→ constant we derive for 1 ≤ j ≤ q + 1,

$$\begin{array}{l}E({\mathcal{L}}_{1,j})={nb}_n\int Q_1[\{s(T_1,t_0)+{\mathbf{X}}_1^T{\mathit{\theta }}_g(T_1)\},\mu ({\mathbf{X}}_1,T_1)]K\{(T_1-t_0)/h\}\times X_{1j}{f}_{\mathbf{x},t}({\mathbf{X}}_1,T_1)d{\mathbf{X}}_1{dT}_1\\ ={(nh)}^{1/2}\int Q_1[\{s(wh+t_0,t_0)+{\mathbf{X}}_1^T{\mathit{\theta }}_g(wh+t_0)\},\mu ({\mathbf{X}}_1,wh+t_0)]\times K(w)X_{1j}{f}_{\mathbf{x},t}({\mathbf{X}}_1,wh+t_0)d{\mathbf{X}}_{1}dw\\ =\{{({nh}^5)}^{1/2}/2\}\int \rho ({\mathbf{X}}_1,wh+t_0)w^2{\mathbf{X}}_i^T{\mathit{\eta }}^{″}(t_0)\times K(w)X_{1j}{f}_{\mathbf{x},t}({\mathbf{X}}_1,wh+t_0)d{\mathbf{X}}_1{dT}_1+o_p\{{({nh}^5)}^{1/2}\}\\ =\{{({nh}^5)}^{1/2}/2\}{\kappa }_2{f}_t(t_0)E[{\mathbf{X}}_i^T{\mathit{\eta }}^{″}(t_0)\rho ({\mathbf{X}}_1,T_1)X_{1j}\mid T_1=t_0]+o_p\{{({nh}^5)}^{1/2}\}.\end{array}$$

Similarly we derive that $E({\mathcal{L}}_{1,j})=\{{({nh}^5)}^{1/2}/2\}{\kappa }_3{f}_t(t_0)E[{\mathbf{X}}_i^T{\mathit{\eta }}^{″}(t_0)\rho ({\mathbf{X}}_1,T_1)X_{1j}\mid T_1=t_0]+o_p\{{({nh}^5)}^{1/2}\}$ for q + 2 ≤ j ≤ 2(q + 1). Hence we have that

$$\begin{array}{l}E({\mathcal{L}}_1)={({nh}^5)}^{1/2}\{f_t(t_0)/2\}\left(\begin{array}{c}{\kappa }_2\\ {\kappa }_3\end{array}\right)\otimes E[\rho ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_1=t_0]{\mathit{\eta }}^{″}(t_0)+o_p\{{({nh}^5)}^{1/2}\}\\ ={({nh}^5)}^{1/2}{\mathbf{B}}_1(t_0)+o_p(1).\end{array}$$

Now we take up var( ). Define Z_i to be the term inside the sum in the definition of . Since for any j, k, Z_j is independent of Z_k when j ≠ k, it is straightforward to derive

$$cov({\mathcal{L}}_{1,j}{\mathcal{L}}_{1,k})={nb}_n^2[E(Z_{1j}{Z}_{1k})-E(Z_{1j})E(Z_{1k})].$$

Using similar argument to those in the computation of E( ), we derive that E(Z_1j) = O_p(h³) for any j and thus ${nb}_n^{2}E(Z_{1j})E(Z_{1k})=o_p(1)$. Next, for 1 ≤ j, k ≤ (q + 1),

$$\begin{array}{l}{nb}_n^{2}E(Z_{1j}{Z}_{1k})={nb}_n^{2}E[Q_1^2[\{s(T_1,t_0)+{\mathbf{X}}_1^T{\mathit{\theta }}_g(T_1)\},Y_1]K^2\{(T_1-t_0)/h\}{X}_{1,j}{X}_{1k}]\\ ={\nu }_0{f}_t(t_0)E\{\gamma ({\mathbf{X}}_1,T_1)X_{1j}{X}_{1k}\mid T_1=t_0\}+o_p(1),\end{array}$$

where γ(x, t₀) = var(Y₁|X = x, T₁ = t₀)/[V {μ(x, t₀)}g′{μ(x, t₀)}]². Similarly we obtain

$$var({\mathcal{L}}_1)=f_t(t_0)\mathbf{R}\otimes E\{\gamma ({\mathbf{X}}_1,T_1){\mathbf{X}}_1{\mathbf{X}}_1^T\mid T_1=t_0\}+o_p(1)={\mathbf{V}}_2(t_0)+o_p(1),$$ (A4)

where R is a 2 × 2 matrix with elements R_jk = ν_j+k−2.

The final step of the argument is to use the Cramer-Wold device, that is to show that for any vector d

$${\{{\mathbf{d}}^{T}var({\mathcal{L}}_1)\mathbf{d}\}}^{-1/2}\{{\mathbf{d}}^T{\mathcal{L}}_1-{\mathbf{d}}^{T}E({\mathcal{L}}_1)\}{\to }^d\mathrm{N}(0,1).$$

We only need to check Lyapounov’s condition. To this end, recall that ${\mathcal{L}}_1=b_n{\sum }_{i=1}^n{\mathbf{Z}}_i$. It is sufficient to show that ${nb}_n^{3}E({\mid {\mathbf{d}}^T{\mathbf{Z}}_1\mid }^3)\to 0$. This result follows from a similar argument as that in (A1). Hence we obtain var( )^−1/2 { − E( )} → N(0, 1).

Combining (A3) – (A4) we obtain

$$\widehat{\mathit{\delta }}-{({nh}^5)}^{1/2}{\mathbf{V}}^{-1}(t_0){\mathbf{B}}_1(t_0){\to }^p\mathrm{N}(\mathbf{0},{\mathbf{V}}_1^{-1}{\mathbf{V}}_2{\mathbf{V}}_1^{-1})$$

Theorem 2.1 now follows from the definition of δ.

A.3. Proof of Theorem 2.2

Recall the definition of Q(β; h, t, α) from Section 2.2 for various guides. Denote β̂(t, α) to be the maximizer of Q(β; h, t, α). When an estimated guide α̂ is used, it suffices to show that ||β̂(t, α̂) − β̂(t, α^*)|| converges to zero in probability with a rate faster than (nh)_1/2, the convergence rate shown in Theorem 2.1 with a fixed guide.

We start by observing two facts. First, n^-1/2||α̂ − α^*|| = O_p(1), which can be shown using the second set of assumptions at the end of Section A.1. Second, using the assumption that Q₂(u, v) < 0 for each u and v in the range of the response variable, it is evident that both Q(β; h, t, α^*) and Q(β; h, t, α̂) are strictly concave in β. Thus we have

$$n^{-1}Q(\mathit{\beta };h,t,\widehat{\mathit{\alpha }})=n^{-1}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast })+O_p(n^{-1/2});$$ (A5)

$$n^{-1}\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,\widehat{\mathit{\alpha }})=n^{-1}\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast })+O_p(n^{-1/2});$$ (A6)

$$n^{-1}\frac{{\partial }^2}{\partial \mathit{\beta }\partial {\mathit{\beta }}^T}Q(\mathit{\beta };h,t,\widehat{\mathit{\alpha }})=n^{-1}\frac{{\partial }^2}{\partial \mathit{\beta }\partial {\mathit{\beta }}^T}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast })+O_p(n^{-1/2}),$$ (A7)

for any β, where the last two equations hold entry-wise. Thus it follows from (A5) that ||β̂(t, α̂) − β̂(t, α^*)|| = o_p(1).

Using (A6), and a Taylor’s expansion, we have

$$\begin{array}{l}0=n^{-1}\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,\widehat{\mathit{\alpha }}){\mid }_{\mathit{\beta }=\widehat{\mathit{\beta }}(t,\widehat{\mathit{\alpha }})}\\ =n^{-1}\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast }){\mid }_{\mathit{\beta }=\widehat{\mathit{\beta }}(t,\widehat{\mathit{\alpha }})}+O_p(n^{-1/2})\\ =n^{-1}\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast }){\mid }_{\mathit{\beta }=\widehat{\mathit{\beta }}(t,{\mathit{\alpha }}^{\ast })}+n^{-1}\frac{{\partial }^2}{\partial \mathit{\beta }\partial {\mathit{\beta }}^T}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast }){\mid }_{\mathit{\beta }=\widehat{\mathit{\beta }}(t,{\mathit{\alpha }}^{\ast })}[\widehat{\mathit{\beta }}(t,\widehat{\mathit{\alpha }})-\widehat{\mathit{\beta }}(t,{\mathit{\alpha }}^{\ast })]+O_p(n^{-1/2}).\end{array}$$

Recall that by definition of β̂(t, α^*) we have $\frac{\partial }{\partial \mathit{\beta }}Q(\mathit{\beta };h,t,{\mathit{\alpha }}^{\ast }){\mid }_{\mathit{\beta }=\widehat{\mathit{\beta }}(t,{\mathit{\alpha }}^{\ast })}=0$. Also, using similar argument used to derive (A3), the Frobenius norm of the second derivative term in the expansion above can be shown to be O_p(1). Thus we have that

$$\Vert \widehat{\mathit{\beta }}(t,\widehat{\mathit{\alpha }})-\widehat{\mathit{\beta }}(t,{\mathit{\alpha }}^{\ast })\Vert =O_p(n^{-1/2}).$$

The rate n^1/2 is much faster than the rate (nh)^1/2 shown in Theorem 2.1, and hence the result follows.