Polynomial Spline Estimation for A Generalized Additive Coefficient Model

Abstract

We study a semiparametric generalized additive coefficient model, in which linear predictors in the conventional generalized linear models is generalized to unknown functions depending on certain covariates, and approximate the nonparametric functions by using polynomial spline. The asymptotic expansion with optimal rates of convergence for the estimators of the nonparametric part is established. Semiparametric generalized likelihood ratio test is also proposed to check if a nonparametric coefficient can be simplified as a parametric one. A conditional bootstrap version is suggested to approximate the distribution of the test under the null hypothesis. Extensive Monte Carlo simulation studies are conducted to examine the finite sample performance of the proposed methods. We further apply the proposed model and methods to a data set from a human visceral Leishmaniasis (HVL) study conduced in Brazil from 1994 to 1997. Numerical results outperform the traditional generalized linear model and the proposed generalized additive coefficient model is preferable.

1 INTRODUCTION

The most common model used in analyzing the relationship between a discrete response variable and covariates is the generalized linear model (McCullagh & Nelder, 1989). With a given link function, it models the relationship between the dependent and explanatory variables through a linear function form. However, many data arising in a variety of disciplines, such as, economics, political science, geography and epidemiology, require more flexible forms than the usual linearity. Recently, many non- and semi-parametric models have been proposed to relax the strict linear assumption in the generalized linear models, such as, the generalized additive models (Hastie & Tibshirani, 1990; Härdle et al., 2004), the generalized varying coefficient models (Cai et al., 2000), the generalized partially linear models (Green & Silverman, 1994; Härdle et al., 2000; Liang & Ren, 2005), and the generalized partially linear single index models (Carroll et al., 1997).

In this paper, we propose a new semiparametric model, namely the generalized additive coefficient model (GACM), which is an extension of varying coefficient models (Hastie & Tibshirani, 1993). Similar to the generalized varying coefficient model, it allows the coefficients of the linear covariates to depend on certain covariates, called tuning variables. But it further imposes an additive function form on the coefficient functions to circumvents the so-called “curse-of-dimensionality” problem when the dimension of the tuning variables is large. As seen in Section 2, the proposed GACM is flexible enough to include the aforementioned non- and semiparametric models as special cases.

A motivation of this study comes from an analysis of an epidemiological data set. It consists of the human visceral Leishmaniasis (HVL) case numbers in 117 health zones in Belo Horizonte, Brazil from years 1994 to 1997. HVL is mainly a rural disease which has become prevalent in recent years in Brazilian urban areas. The first human case of HVL was recorded in March 1989 in Sabará, a municipality located in the Belo Horizonte metropolitan region. Afterwards, in spite of undertaken control actions, the disease spread into the city from the northeast. The annual human cases recorded in years 1994, 1995, 1996 are 29, 46, and 45 respectively. A total of 40 cases were already reported only in the first semester of 1997. As argued in Assunção (2003), the small number of cases in each area produced very unstable rates, preventing a more focused public health action. One of the main interests of the study is to model the disease diffusion over time to better monitor the disease and allocate resources for disease control. A possible approach is to model the HVL case numbers using a traditional Poisson regression model with a polynomial time trend, see model (8). But Belo Horizonte is a large Brazilian city with great social, econometric, and geological diversity. Also the disease first appeared in the northwest, then spread into the city afterwards. Thus the dynamic of the disease progress over time is different over the whole space. A traditional poisson model with constant coefficients, such as model (8), over the whole space may not be able to capture this spatially varying phenomena. But this can be incorporated into the generalized additive coefficient model, by allowing the coefficients of linear covariats to vary smoothly with the location indexes (the latitude, and longitude), see model (7). Our analysis in Section 6 shows that the GACM out-performers the generalized linear model in terms of both estimation and prediction.

In the least squares setting, Xue & Yang (2006 a & b) considered the estimation of the additive coefficient model for gaussian data using both kernel and polynomial spline methods. In contrast, this paper studies the estimation, also testing of the model for non-gaussian data through maximizing the likelihood with polynomial spline smoothing. The convergence results of the maximum likelihood estimates in this paper are similar to those for regression established by Xue & Yang (2006b). But as Huang (1998) pointed out, it’s more technically challenging to establish the rate of convergence for the maximum likelihood estimates, since it cannot be viewed simply as an orthogonal projection due to its nonlinear structure. Another contribution of this paper is to propose an efficient testing procedure for the coefficient functions by combing the polynomial spline smoothing with the conditional bootstrapping.

The use of the polynomial spline smoothing in the generalized nonparametric models has been investigated in various contexts. Stone (1986) first obtained the rate of convergence of the polynomial spline estimates for the generalized additive model. Stone (1994) and Huang (1998) focused on the polynomial spline estimation of the generalized functional ANOVA model, while Huang et. al. (2000) and Huang & Liu (2006) considered the functional ANOVA model and the single-index model in the proportional hazard regression via maximum partial likelihood estimation respectively. Polynomial spline smoothing is a global smoothing method, which approximates the unknown functions via polynomial splines characterized by only a finite number of coefficients. After the spline basis is chosen, the coefficients can be estimated by an efficient one-step procedure of maximizing the likelihood function. It is computationally cheaper in contrast to the kernel based methods, where the maximizing has to be conducted repeatedly at every local data points. Thus the application of the polynomial spline smoothing in current context is particularly computationally efficient.

This paper is organized as follows. Section 2 introduces the GACM. Section 3 gives an efficient polynomial spline estimation method for the proposed model. Mean square (or L₂) convergence results are established for the estimators under mild assumptions. Section 4 discusses a testing procedure of the coefficient functions via conditional bootstrap approach. The simulation studies and an application of the proposed methods in a real data example are included in Sections 5 and 6 respectively. The technical lemmas and proofs are given in the appendix.

2 THE MODEL

In our definition of the generalized regression models, we follow the notation in Stone (1986, 1994), and Huang (1998). The set-up involves an exponential family of distributions of the form

$$\exp (B\left(\eta \right)y-C\left(\eta \right))\rho \left(dy\right),$$

where B (·) and C (·) are known functions with $C\left(\eta \right)=\log {\int }_{\mathcal{R}}\exp \left[B\left(\eta \right)y\right]\rho \left(dy\right)$, and ρ is a non-zero measure defined on $\mathcal{R}$ that is not concentrated on a single point. Correspondingly, the mean of the distribution μ = A (η) = C’ (η) / B’ (η), where B’ (·) and C’ (·) are the first-order derivatives of B (·) and C (·) respectively. Equivalently, η = A⁻¹ (μ) with the function A⁻¹ being the link function.

Consider a random vector (Y, X, T), in which Y is a real-valued response variable, and (X, T) are predictor variables with X = (X₁, . . . , X_d2)^T, T = (T₁, . . . , T_d1)^T. The conditional distribution of Y given (X, T) is connected to the above exponential family distribution through the assumption that

$$E(Y\mid \mathbf{X}=\mathbf{x},\mathbf{T}=\mathbf{t})=A\left(\eta \left(x,t\right)\right),\eta \left(x,t\right)=\sum _{l=1}^{d_1}\left\{{\alpha }_{l0}+\sum _{s=1}^{d_2}{\alpha }_{ls}\left(x_s\right)\right\}{t}_1.$$ (1)

The model (1) is called the generalized additive coefficient model (GACM). To assure the components in (1) are identifiable, we impose restriction E {α_ls (X_s)} = 0, for l = 1, . . . , d₁, s = 1, . . . , d₂. As in most work on nonparametric smoothing, estimation of the functions ${\left\{{\alpha }_{ls}\right\}}_{l=1,s=1}^{d_1,d_2}$ is conducted on a compact support. Without loss of generality, let the compact set be χ = [0, 1]^d₂.

The proposed GACM in (1) is quite general. It is flexible enough to cover a variety of situations. For example, when d₂ = 0, or equivalently, there is no predictors X, (1) becomes the generalized linear models. When d₂ = 1, (1) becomes the generalized varying coefficient model (Cai et al., 2000). When T₁ = . . . = T_d1 =constant, (1) becomes the generalized additive model (Hastie & Tibshirani, 1990; Härdle et al., 2004).

Similar to Huang (1998), if the conditional distribution of Y given X = x, T = t follows the exponential family distribution described above with η = η (x, t), then the assumption (1) is satisfied and the log-likelihood function is given by l (h, X, T, Y) = B (h (X, T)) Y − C (h (X, T)), for any function h defined on $\chi \times {\mathcal{R}}^{d_1}$. If the conditional distribution of Y given X = x, T = t does not follow the exponential family distribution, we can think l (h, X, T, Y) as a pseudo-log-likelihood. For simplicity, we refer to both cases as the log-likelihood function.

3 POLYNOMIAL SPLINE ESTIMATION

We propose to estimate the nonparametric functions in model (1) using the polynomial spline smoothing method. It involves an approximation of the nonparametric functions ${\left\{{\alpha }_{ls}(\cdot )\right\}}_{l=1,s=1}^{d_1,d_2}$ using polynomial splines.

For each of the tuning variable directions, i.e., s = 1, . . . , d₂, we introduce a knot sequence k_s,n on [0, 1], with N_n interior knots, k_s,n = {0 = v_s,0 < v_s,1 < · · · < v_s,Nn < v_s,Nn+1 = 1}. For any nonnegative integer p, we denote φ_s = φ^p([0, 1], k_s,n), the space of functions, whose element

1. is a polynomial of degree p (or less) on each of the intervals [v_s,i, v_s,i+1) for i = 0, . . . , N_n−1, and [v_s,Nn, v_s,Nn+1],

2. and is p − 1 continuously differentiable on [0, 1] if p ≥ 1.

A function that satisfies (i) and (ii) is called a polynomial spline, a piecewise polynomial connected smoothly on the interior knots. For example, a polynomial spline with degree p = 0 is a piecewise constant function, and a polynomial spline with degree p = 1 is a piecewise linear function and continuous on [0, 1]. The polynomial spline space φ_s is determined by the degree of the polynomial p and the knot sequence k_s,n. Let h_s = h_s,n = max_i=0,...,Nn |v_s,i+1 − v_s,i|, which is called the mash size of k_s,n and can be understood as the smoothness parameter like the bandwidth in the local polynomial context. Define h = max_s=1,...,d2 h_s, which measures the overall smoothness.

To consistently estimate functions ${\left\{{\alpha }_{ls}\left(x_s\right)\right\}}_{l=1,s=1}^{d_1,d_2}$ in (1) which are centered with E (α_ls (X_s)) = 0, we introduce empirically centered polynomial splines,

$${\phi }_s^{0,n}=\left\{g_s:g_s\in {\phi }_s,\sum _{i=1}^n{g}_s\left(X_{is}\right)∕n=0\right\}.$$

The basis of ${\phi }_s^{0,n}$ can be conveniently constructed. For example, we have used the empirically centered truncated power basis in the implementation, i.e.

$${\left\{B_{sj}=b_{sj}-\sum _{i=1}^n{b}_{sj}\left(X_{is}\right)∕n\right\}}_{j=1}^{J_n},$$

where J_n = N_n + p, and {b_s1, . . . , b_sJn} is the truncated power basis given as

$$\{x_s,\dots ,x_s^p,{(x_s-v_{s,1})}_{+}^p,\dots ,{(x_s-v_s,N_s)}_{+}^p\},$$

in which ${\left(x\right)}_{+}^p={\left(x_{+}\right)}^p$. If the functions ${\left\{{\alpha }_{ls}\left(x_s\right)\right\}}_{l=1,s=1}^{d_1,d_2}$ are smooth enough, then one can approximate them by polynomial splines ${\{g_{ls}\left(x_s\right)\in {\phi }_s^{0,n}\}}_{l=1,s=1}^{d_1,d_2}$. That is, for each l = 1, . . . , d₁, s = 1, . . . , d₂, one has

$${\alpha }_{ls}\left(x_s\right)\approx g_{ls}\left(x_s\right)=\sum _{j=1}^{J_n}{c}_{ls,j}{B}_{sj}\left(x_s\right),$$

with a set of coefficients ${\left\{c_{ls,j}\right\}}_{j=1}^{J_n}$. Denote l(η(X, T), Y) = B[η(X, T)]Y − C[η(X, T)], and $l_n\left(\eta \right)=\frac{1}{n}{\sum }_{i=1}^{n}l(\eta ({\mathbf{X}}_i,{\mathbf{T}}_i),Y_i)$ for any $\eta (\mathbf{X},\mathbf{T})={\sum }_{l=1}^{d_1}\{{\alpha }_{l0}+{\sum }_{s=1}^{d_2}{\alpha }_{ls}\left(X_s\right)\}{T}_l$. Then the log-likelihood function l_n (η) can be approximated by

$$\begin{array}{cc}\hfill l_n\left(\eta \right)& \approx l_n({\alpha }_0,\mathbf{c})=\frac{1}{n}\sum _{i=1}^{n}l\left\{\sum _{l=1}^{d_1}\left\{{\alpha }_{l0}+\sum _{s=1}^{d_2}{g}_{ls}\left(X_{is}\right)\right\}{T}_{il},Y_i\right\}\hfill \\ \hfill & =\frac{1}{n}\sum _{i=1}^{n}l\left\{\sum _{l=1}^{d_1}\left\{{\alpha }_{l0}+\sum _{s=1}^{d_2}\sum _{j=1}^{J_n}{c}_{ls,j}{B}_{sj}\left(X_{is}\right)\right\}{T}_{il},Y_i\right\},\hfill \end{array}$$ (2)

in which the coefficients α₀ = (α₁₀, . . . , α_d10)^T and $\mathbf{c}={\left\{c_{ls,j}\right\}}_{l=1,s=1,j=1}^{d_1,d_2,J_n}$ can be solved by maximizing the log-likelihood function, i.e.

$$({\widehat{\alpha }}_0,\widehat{\mathbf{c}})=\arg \underset{{\alpha }_0,\mathbf{c}}{\max }{l}_n({\alpha }_0,\mathbf{c}).$$ (3)

Then the resulting estimator of the functions are given as

$${\widehat{\alpha }}_{ls}\left(x_s\right)=\sum _{j=1}^{J_n}{\widehat{c}}_{ls,j}{B}_{sj}\left(x_s\right),$$

for l = 1, . . . , d₁, s = 1, . . . , d₂. As a result,

$$\widehat{\eta }\left(x,t\right)=\sum _{l=1}^{d_1}\left\{{\widehat{\alpha }}_{l0}+\sum _{s=1}^{d_2}{\widehat{\alpha }}_{ls}\left(x_s\right)\right\}{t}_l.$$ (4)

The maximization of (3) can be easily carried out using existing software for generalized linear models. Furthermore, only one maximum likelihood procedure is needed to estimate all the components in the coefficient function, which is much more computationally efficient than the kernel based method where one needs to perform the maximum likelihood estimation at each local point. On the other hand, the next theorem shows that the polynomial spline estimators enjoy the same optimal rate of convergence as the kernel estimators. In the following, ||·|| denotes the theoretical L₂-norm defined by (A.1) in the appendix.

Theorem 1

Under the assumptions (C1)-(C8) in the Appendix, one has,

$$\mid \mid \widehat{\eta }-\eta \mid \mid =O_p(h^{p+1}+\sqrt{1∕nh}),$$

and for l = 1, . . . , d₁, s = 1, . . . , d₂,

$$\mid {\widehat{\alpha }}_{l0}-{\alpha }_{l0}\mid =O_p(h^{p+1}+\sqrt{1∕nh}),\mid \mid {\widehat{\alpha }}_{ls}-{\alpha }_{ls}\mid \mid =O_p(h^{p+1}+\sqrt{1∕nh}).$$

Theorem 1 shows the mean square (or L₂) consistency of polynomial spline estimators. When the smoothing parameter h takes the optimal order of n^−1/(2p+3), ||α̂_ls − α_ls||₂ = O_p (n^{−(p+1)/(2p+3)}), which is the optimal rate of convergence for univariate nonparametric functions. As a result, each of the d₂ dimensional coefficient functions ${\alpha }_l\left(\mathbf{x}\right)={\alpha }_{l0}+{\sum }_{s=1}^{d_2}{\alpha }_{ls}\left(x_s\right)$ is also estimated at the univariate optimal rate. Therefore the proposed GACM effectively avoids the “curse-of-dimensionality” by assuming an additive structure on the coefficient functions.

3.1 Knot number selection

The proposed polynomial spline estimation procedure crucially depends on appropriate choice of knot sequence ${\left\{k_{s,n}\right\}}_{s=1}^{d_2}$, and in particular, on the number of interior knot N_n. Here we propose to select N_n using an AIC procedure. For knot location, we use either equally-spaced knots or quantile knots (sample quantiles with the same number of observations between any two adjacent knots). The similar procedure was also used in Huang et al. (2002), and Xue & Yang (2006b).

According to Theorem 1, the optimal order of N_n is n^1/(2p+3). Thus we propose to choose the optimal knot number, $N_n^{\mathrm{opt}}$, from a neighborhood of n^1/(2p+3). For our examples in Sections 5 and 6, we have used [0.5N_r, min(5N_r, Tb)], where N_r = ceiling(n^1/(2p+3)), and Tb = {n/ (4d₁) − 1} /d₂ to ensure the total number of parameters in (2) is less than n/4. Let η̂_Nn (·) be the estimator of η (·) with number of knots N_n, and the resulting log-likelihood function $l_n\left(N_n\right)=\frac{1}{n}{\sum }_{i=1}^{n}l({\widehat{\eta }}_{N_n}({\mathbf{X}}_i,{\mathbf{T}}_i),Y_i)$. Let q_n = (1 + d₂J_n) d₁ be the total number of parameters in (2). Then the optimal knot number, $N_n^{\mathrm{opt}}$, is the one which minimizes the AIC value. That is

$$N_n^{\mathrm{opt}}=\underset{N_n\in [0.5N_r,\min (5N_r,Tb)]}{\mathrm{argmin}}\mathrm{AIC}\left(N_n\right)=\underset{N_n\in [0.5N_r,\min (5N_r,Tb)]}{\mathrm{argmin}}\{-2l_n\left(N_n\right)+2q_n\}.$$

4 HYPOTHESIS TESTING

After fitting GACM (1), a natural question arising is whether the coefficient functions ${\left\{{\alpha }_{ls}\right\}}_{l=1,s=1}^{d_1,d_2}$ are actually varying, or more generally, whether certain parametric models, such as polynomials, fit the nonparametric components. This leads us to consider hypothesis testing problems such as:

$$H_0:{\alpha }_{ls}\left(x_s\right)={\alpha }_{ls}(x_{ls},\theta )\text{versus}{H}_1:{\alpha }_{ls}\left(x_s\right)\ne {\alpha }_{ls}(x_{ls},\theta ),$$

where ${\alpha }_{ls}(x_s,\theta )={\sum }_{k=0}^q{\theta }_k{x}_s^k$, and θ is a vector of unknown parameters in the polynomial function. It includes testing whether the component α_ls is varying, in which α_ls (x_s, θ) = 0. One option is the nonparametric likelihood ratio test statistic (Fan et al., 2001), which is defined as

$$T_n=2\{l_n\left(H_1\right)-l_n\left(H_0\right)\},$$ (5)

in which l_n (H₀) and l_n (H₁) are the log-likelihood functions calculated under the null and alternative hypothesis respectively. To be more specific, under null hypothesis, we model α_ls as a polynomial of degree q and approximate all the other functions in the model with polynomial spline of degree p with p ≥ q. Under alternative hypothesis, all functions in the model are approximated with polynomial spline of degree p. We have used the AIC procedure in subsection (3.1) to choose optimal knot number for the full generalized additive coefficient model under alternative hypothesis. Then the same number of knots is used for estimation of the nonparametric functions in the null model.

Theorem 2

Under the assumptions (C1)-(C8) in the Appendix, one has, under H₀, T_n → 0 in probability as n → ∞; otherwise, there exits δ > 0, such that T_n > δ with probability tending to one.

The result of Theorem 2 suggests to reject H₀ for large T_n. To obtain an appropriate critical value, we approximate the null distribution of T_n using the conditional bootstrap method; see also Cai et al. (2000), and Fan and Huang (2005). Let ${\left\{{\widehat{\alpha }}_{l0}^0\right\}}_{l=1}^{d_1}$, and ${\left\{{\widehat{\alpha }}_{ls}^0\right\}}_{l=1,s=1}^{d_1,d_2}$ be the estimators of the constants and coefficient functions under H₀. Let the resulting estimator of η (x, t) be ${\widehat{\eta }}^0(\mathbf{x},\mathbf{t})={\sum }_{l=1}^{d_1}\{{\widehat{\alpha }}_{l0}^0+{\sum }_{s=1}^{d_2}{\widehat{\alpha }}_{ls}^0\left(x_s\right)\}{t}_l$. In the conditional bootstrap procedure, a total of B bootstrap samples are generated. In our examples given in Sections 5 and 6, we have used B = 500. In each of the samples (b = 1, . . . , B), the value of independent variables (X_i, T_i) are kept the same as the observed ones, while a bootstrap sample $Y_i^b$ is generated from the distribution of Y, with η (x, t) being ${\widehat{\eta }}^0({\mathbf{X}}_i,{\mathbf{T}}_i)$. Then a test statistics $T_n^b$ in (5) is computed from the bootstrap sample ${({\mathbf{X}}_i,{\mathbf{T}}_i,Y_i^b)}_{i=1}^n$. In the implementation, the AIC knot selection procedure for the alternative model is performed for each bootstrap sample. The distribution of ${\left\{T_n^b\right\}}_{b=1}^B$ is used to approximate the distribution of the test statistic T_n. In particular, for a given level of significance α, the (1 − α) 100% percentile of ${\left\{T_n^b\right\}}_{b=1}^B$ is used as the critical value.

5 SIMULATION STUDY

In this section, we investigate the finite-sample performance of the proposed estimation and testing methods through two simulation studies. We use the averaged integrated squared error (AISE) to evaluate the performance of the function estimators ${\left\{{\widehat{\alpha }}_{ls}(\cdot )\right\}}_{l,s=1}^{d_1,d_2}$, which is defined as

$$\mathrm{AISE}\left({\widehat{\alpha }}_{ls}\right)=\frac{1}{\mathrm{nrep}}\sum _{r=1}^{\mathrm{nrep}}\mathrm{ISE}\left({\widehat{\alpha }}_{ls}^r\right),\mathrm{ISE}\left({\widehat{\alpha }}_{ls}^r\right)=\frac{1}{n_{\mathrm{grid}}}\sum _{m=1}^{n_{\mathrm{grid}}}{\{{\alpha }_{ls}\left(x_{ms}\right)-{\widehat{\alpha }}_{ls}^r\left(x_{ms}\right)\}}^2,$$

where ${\widehat{\alpha }}_{ls}^r(\cdot )$ denotes the estimator of α_ls(·) in the α_ls (·) in the r-th replication for r = 1, . . . , nrep, and ${\left\{{(x_{m1},\dots ,x_{m{d}_2})}^T\right\}}_{m=1}^{n_{\mathrm{grid}}}$ are the grid points where the nonparametric functions are evaluated. In both examples, we have used the sample size n = 250, 500, 750, and the number of replications nrep = 1000. The nonparametric functions α_ls (·) are all evaluated on a grid of equally-spaced points x_ms, m = 1, . . . , ngrid with x_1s = 0.025, x_ngrid,s = 0.975, and ngrid = 96.

5.1 Logistic regression

Data sets are generated from a logistic regression model where the binary response variable Y_i has the distribution

$$\mathrm{logit}P(Y_i=1\mid {\mathbf{X}}_i={\mathbf{x}}_i,{\mathbf{T}}_i={\mathbf{t}}_i)=\sum _{l=1}^2\left\{{\alpha }_{l0}+\sum _{s=1}^2{\alpha }_{ls}\left(x_s\right)\right\}{t}_l,$$

in which α₁₀ = 1, α₂₀ = 0, α₁₁ (x) = α₂₁ (x) = sin (2πx), α₁₂ (x) = 0, and α₂₂ (x) = 2x − 1. The covariates T_i = (T_i1, T_i2)^T, and X_i = (X_i1, X_i2)^T are independently generated from the standard bivariate normal and Uniform([0, 1]²) respectively.

We have applied the proposed polynomial spline estimation method with both linear spline (p = 1) and cubic spline (p = 3). Estimation with other degrees such as quadratic spline (p = 2) can also be used, but give similar findings. We have used the AIC procedure to select the number of knots that are evenly placed over the ranges of x_is, for each s = 1, . . . , d₂. Table 1 summarizes the means and standard errors (in the parentheses) of {α̂_l0}_l=1,2, and the AISEs of ${\left\{{\widehat{\alpha }}_{ls}(\cdot )\right\}}_{l=1,2}^{s=1,2}$ from both linear and cubic spline estimations. It shows that two spline fits are generally comparable with cubic spline slightly better than the linear spline in smaller sample sizes. In both cases, the standard errors of the constant estimators and the AISEs of the function estimators decrease as the sample size n increases, which confirms Theorem 1. The typical estimated curves (whose ISE is the median in the 1000 replications) from linear polynomial spline estimation are plotted in Figure 1, together with the point-wise 95% confidence intervals when n = 500. It clearly shows that the linear spline fits are reasonably good.

Table 1.

Logistic regression example: the means and standard errors (in parentheses) of α̂₁₀, and α̂₂₀ and the AISEs of α̂₁₁ (·), α̂₁₂ (·), α̂₂₁ (·), and α̂₂₂ (·) from 1000 replications

Methods	n	α10 = 1	α20 = 0	α 11	α 12	α 21	α 22
Linear spline	250	1.169(0.321)	−0.009(0.251)	0.4234	0.2182	0.3256	0.2684
	500	1.068(0.157)	0.005(0.134)	0.0953	0.0726	0.0804	0.0686
	750	1.047(0.126)	0.003(0.108)	0.0679	0.0451	0.0546	0.0418
Cubic spline	250	1.197(0.304)	0.002(0.236)	0.3531	0.2165	0.2429	0.2689
	500	1.082(0.159)	0.007(0.137)	0.0954	0.0799	0.0751	0.0682
	750	1.066(0.125)	0.001(0.111)	0.0688	0.0495	0.0527	0.0467

Figure 1.

Logistic regression example: plots of the typical estimated coefficient functions using linear polynomial spline method: (a) α₁₁; (b) α₁₂; (c) α₂₁; (d) α₂₂. In each plot, the solid curve represents the true curve, the dashed curve is the typical estimated curve with n = 250, the dotted curve is with n = 500 and the dotdash curve is with n = 750. The two long-dashed curves are the point-wise 95% confidence intervals when n = 500.

Next we examine the proposed testing procedure and consider the following hypothesis

$$H_0:{\alpha }_{12}(\cdot )=0,v.s.H_1:{\alpha }_{12}(\cdot )\ne 0.$$ (6)

The powers are evaluated under a sequence of the alternative models, H₁ : α₁₂ (x) = λ sin(2πx), where λ controls the degree of departure from the null hypothesis, with λ = 0 corresponding to H₀. The value λ is taken to be a grid of equally spaced points on [0, 1.5]. Based on 1000 replications for the sample size n = 250, 500 and 750, Figure 2 plots the power functions with significant level α = 0.05. It shows that the power increases to 1 rapidly as λ increases. The powers at λ = 0 are 0.054, 0.056, 0.051 for n = 250, 500 and 750 respectively, which are all close to the corresponding significant level.

Figure 2.

Logistic example: the power function of the test statistics T_n is plotted against λ, for sample sizes n = 250 (solid curve), 500 (dashed curve), and 750 (dotted curve). The significant level is 0.05.

5.2 Poisson regression

In this example, we consider a Poisson regression model with

$$\log E(Y_i\mid {\mathbf{X}}_i={\mathbf{x}}_i,{\mathbf{T}}_i={\mathbf{t}}_i)=\sum _{l=1}^2\left\{{\alpha }_{l0}+\sum _{s=1}^2{\alpha }_{ls}\left(x_s\right)\right\}{t}_l,$$

where different forms of coefficient functions are considered, with α_l0 = 1, α₂₀ = 0, and α₁₁ (x) = 4x (1 − x)−2/3, α₁₂ (x) = 0, α₂₁ (x) = sin² (πx)−0.5, α₂₂ (x) = exp (2x − 1) / [exp (1) − exp (−1)]−1/2. The covariates are generated in the same way as in the Logistic regression example.

Similarly as in the Logistic regression example, we have used both linear spline (p = 1) and cubic spline (p = 3) estimations of the coefficient functions. The equally spaced knots are used with the number of interior knots chosen using the AIC procedure. The simulation results are summarized in Table 2, which contains the standard errors (in the parentheses) of {α̂_l0}_l=1,2, and the AISEs of ${\left\{{\widehat{\alpha }}_{ls}(\cdot )\right\}}_{l=1,2}^{s=1,2}$ from two spline fits. Similarly as in the Logistic regression example, Table 2 shows the convergence of both {α̂_l0}_l=1,2, and ${\left\{{\widehat{\alpha }}_{ls}(\cdot )\right\}}_{l=1,2}^{s=1,2}$, as n increases. It again collaborates Theorem 1. The typical estimated curves from linear spline method with their point-wise 95% confidence intervals when n = 500 in Figure 3 show that the proposed spline method gives reasonable estimators of the coefficient functions. We also studied the performance of the proposed testing procedure for this Poisson regression. The same hypothesis (6) in the Logistic regression example are considered. The powers are evaluated under the alternative models, H₁ : α₁₂ (x) = λ [4x (1 − x) − 2/3], with λ being a grid of equally spaced points on [0, 1.5]. Based on 1000 replications for the sample size n = 250, 500, and 750, Figure 4 plots the power functions with significant levels α = 0.05. The powers at λ = 0 are 0.045, 0.053, 0.054 respectively, which are close to the significant levels.

Table 2.

Poisson regression example: the means and standard errors (in parentheses) of α̂₁₀, and α̂₂₀ and the AISEs of α̂₁₁ (·), α̂₁₂ (·), α̂₂₁ (·), and α̂₂₂ (·) using linear and cubic spline estimations from 1000 replications

Methods	n	α10 = 1	α20 = 0	α 11	α 12	α 21	α 22
Linear spline	250	0.9923(0.054)	0.0013(0.061)	0.0099	0.0084	0.0168	0.0119
	500	0.9963(0.033)	0.0022(0.041)	0.0047	0.0036	0.0084	0.0054
	750	0.9957(0.027)	0.0005(0.033)	0.0030	0.0022	0.0056	0.0034
Cubic spline	250	0.9896(0.053)	0.0016(0.060)	0.0092	0.0091	0.0134	0.0112
	500	0.9945(0.033)	0.0017(0.039)	0.0041	0.0032	0.0053	0.0048
	750	0.9941(0.027)	0.0009(0.032)	0.0031	0.0024	0.0042	0.0037

Figure 3.

Poisson regression example: plots of the typical estimated coefficient functions using linear polynomial spline method: (a) α₁₁; (b) α₁₂; (c) α₂₁; (d) α₂₂. In each plot, the solid curve represents the true curve, the dashed curve is the typical estimated curve with n = 250, the dotted curve is with n = 500 and the dotdash curve is with n = 750. The two long-dashed curves are the point-wise 95% confidence intervals when n = 500.

Figure 4.

Poisson regression example: the power function of the test statistics T_n is plotted against λ, for sample sizes n = 250 (solid curve), 500 (dashed curve), and 750 (dotted curve). The significant level is 0.05.

6 REAL DATA ANALYSIS

In this section we apply the proposed GACM to analyze the data set from the HVL study introduced in Section 1. The data consists of the annual number of human HVL cases and total population count for each of the 117 zones and each of the periods of 1994/1995, 1995/1996, and 1996/1997. A period comprises the second semester of a year (starting in July 1st) and the first semester of the following year (ending in June 30). For more information of the data, see Assunção et al. (2001). Belo Horizonte is a large Brazilian city with more than 2 million inhabitants. The spatial impacts are not necessarily homogenous over the whole area. Assunção et al. (2001) and Assunção (2003) took the varying spatial effect into account and used the Bayesian spatial varying parameter model to study the diffusion of the disease. Motivated by their analysis, we model the varying spatial effect by using the generalized additive coefficient model as follows.

Let y_it be the annual counts of cases and P_it the risk population in each ozone i, i = 1, . . . , 117, for three years t, t = 0, 1, 2. Time indexes t = 0, 1, 2 represents the periods of 1994/1995, 1995/1996, and 1996/1997 respectively. Similarly as in Assunção et al. (2001), and Assunç ão (2003), we assume, conditional on the relative risk exp (λ_it), the counts are independently distributed according to a Poisson distribution with mean P_it exp (λ_it). A second degree polynomial is assumed on λ_it to model the time trend. To allow for varying spatial effects, we further allow the coefficients of the polynomial terms to vary with the spatial coordinates of each zone (x_i1, x_i2). To be more specific, we assume

$${\lambda }_{it}=\sum _{j=0}^2[{\alpha }_{j0}+{\alpha }_{j1}\left(x_{i1}\right)+{\alpha }_{j2}\left(x_{i2}\right)]t^j.$$ (7)

Model (7) allows the time profile to vary smoothly over space, thus effectively models the space-time interaction. The coefficient functions in (7) are estimated using both linear spline (p = 1) and quadratic spline (p = 2) with knot numbers selected by the AIC procedure as in subsection 3.1. Figure 5 plots the estimated coefficient functions. It shows that two spline fits are very close. Therefore, for simplicity, we only report the results using linear spline in what follows. For comparison, we also consider a standard Poisson regression model with constant coefficients, i.e.,

$${\lambda }_{it}=c_0+c_{1}t+c_2{t}^2.$$ (8)

Figure 5.

The estimated coefficient functions in model (7) using linear splines (dashed curve) and quadratic splines (solid curve). Plotted are (a) α̂₀₁, (b) α̂₀₂, (c) α̂₁₁, (d) α̂₁₂, (e) α̂₂₁, (f) α̂₂₂.

Fits are measured by their Akaike’s Information Criterion (AIC), which is the minus twice the maximized log-likelihood plus twice the number of parameters. Models (7) and (8) give AIC values 470.98 and 626.88 respectively, which indicates that model (7) gives a better fit even with model complexity taken into account. Figure 6 graphically compares the residuals from two models, where the residuals are defined as $R_{\text{it}}=(y_{\text{it}}-{\widehat{y}}_{\text{it}})∕\sqrt{{\widehat{y}}_{\text{it}}}$ with y_it and ŷ_it being the observed and estimated annual count of cases for ith zone and tth year. We also compare the models by their prediction performances. We randomly select 15 zones from the 117 health zones. The observations taken at the last time period 1996/1997 from the selected 15 zones are left out for prediction, while the remaining observations are used for estimation. Then the averaged squared prediction errors (ASPE) are calculated. We replicated the prediction procedure ten times. Then the averaged ASPE from ten replications using models (7) and (8) are reported, which are 1.14, and 1.51 respectively. That is, by efficiently taking the varying spatial effect into account, model (7) not only provides better estimation performance, but also improves prediction accuracy than the traditional Poisson regression model (8). Figure 7 plots the estimated HLV rates (per 100 thousands) from model (7) in the health zones for each of the three period.

Figure 6.

Figures (a)-(c) are the residual plots for three periods: 1994/1995 (a), 1995/1996 (b), and 1996/1997 (c). In each plot, triangle and cross denotes the residuals from Models (7), and (8) respectively.

Figure 7.

The estimated HLV rates (per 100 thousand) in the zones of Belo Horizonte using Model (7). The plots from the left to right corresponds to periods: 1994/1995, 1995/1996, and 1996/1997 respectively.

Finally, one may ask whether the coefficient functions ${\left\{{\alpha }_{ls}\right\}}_{l=0,s=1}^{2,2}$ in model (7) are all significantly different from zero, or whether model (7) can be simplified with some of the coefficient functions deleted. For each l = 0, 1, 2 and s = 1, 2, we apply the idea in Section 4 to test the hypothesis: $H_0^{ls}:{\alpha }_{ls}\left(x_s\right)=0$. Based on 1000 bootstrap samples for each hypothesis, all coefficient functions are significantly different from 0 at level 0.05 with the p-values given as < 0.0001, < 0.0001, 0.02, 0.03, 0.01, 0.03 for hypothesis $H_0^{01}$, $H_0^{02}$, $H_0^{11}$, $H_0^{12}$, $H_0^{21}$, $H_0^{22}$ respectively. We therefore conclude that model (7) is more appropriate to fit this data set than model (8), and the improvement is statistically significant. Furthermore, as a referee pointed out, it is also of interest to test the linearity of the unknown coefficient functions in model (7). For each l = 0, 1, 2 and s = 1, 2, consider null hypothesis that ${\tilde{H}}_0^{ls}:{\alpha }_{ls}\left(x_s\right)={\beta }_{0,ls}+{\beta }_{1,ls}{x}_s$ for some coefficients β_0,ls, β_1,ls. The p-values are 0.016, < 0.0001, 0.032, 0.025, 0.169, 0.021 for hypothesis ${\tilde{H}}_0^{01}$, ${\tilde{H}}_0^{02}$, ${\tilde{H}}_0^{11}$, ${\tilde{H}}_0^{12}$, ${\tilde{H}}_0^{21}$, ${\tilde{H}}_0^{22}$ respectively. It suggests that only the coefficient function α₂₁ in model (7) is of linear form at significance level 0.05.

7 CONCLUSIONS

A polynomial spline estimation method together with a generalized likelihood ratio testing procedure have been proposed for the generalized semiparametric additive coefficient model. Theoretical results have been established under very broad assumptions on the data generating process. Based on our experiences in working with both simulated and empirical examples, implementation of the proposed estimation method is as easy and fast as estimating a simple generalized linear model. The estimators’ performance and their prediction power, however, are both promising as Theorem 1 stipulates. These two aspects of the estimators, together with similar desirable properties of the generalized likelihood ratio testing procedure, make them highly recommendable for statistical inference in multivariate regression setting. A third feature, as mentioned in the introduction, is that the procedures of this paper automatically adapts to the generalized additive models (Hastie & Tibshirani, 1990; Härdle et al., 2004), the generalized varying coefficient models (Cai et al., 2000), the generalized partially linear models (Green & Silverman, 1994; Härdle et al., 2000; Liang & Ren, 2005), the generalized partially linear single index models (Carroll et al., 1997), and simple generalized linear models. Hence, all these models can be simultaneously applied to any given data, and the most appropriate one can be selected via generalized likelihood ratio testing procedure.

APPENDIX

A.1 Notation and Assumptions

To formalize the discussion, we introduce two function spaces: the model space $\mathcal{M}$ and the spline approximation space ${\mathcal{M}}_n$. The model space $\mathcal{M}$ is a collection of functions on χ × R^d₁ defined as,

$$\mathcal{M}=\left\{m(\mathbf{x},\mathbf{t})=\sum _{l=1}^{d_1}{\alpha }_l\left(\mathbf{x}\right)t_l,a_l\left(\mathbf{x}\right)={\alpha }_{l0}+\sum _{s=1}^{d_2}{\alpha }_{ls}\left(x_s\right);{\alpha }_{ls}\in {\mathcal{H}}_s^0\right\},$$

where ${\left\{{\alpha }_{l0}\right\}}_{l=1}^{d_1}$ are finite constants, and ${\mathcal{H}}_s^0=\{\alpha \left(x\right):E\left\{\alpha \left(X_s\right)\right\}=0,E\left\{{\alpha }^2\left(X_s\right)\right\}<+\infty \}$, for 1 ≤ s ≤ d₂. Then the predictor function η (x, t) in (1) is modeled as an element of $\mathcal{M}$. Let ${(Y_i,{\mathbf{X}}_i,{\mathbf{T}}_i)}_{i=1}^n$ be a random sample of size n from the distribution of (Y, X, T) . In what follows, denote by E_n the empirical expectation. For functions $m_1,m_2\in \mathcal{M}$, define the theoretical inner product and the empirical inner product respectively as

$$\begin{array}{c}\hfill \langle m_1,m_2\rangle =E\left\{m_1(\mathbf{X},\mathbf{T})m_2(\mathbf{X},\mathbf{T})\right\},\hfill \\ \hfill {\langle m_1,m_2\rangle }_n=E_n\left\{m_1(\mathbf{X},\mathbf{T})m_2(\mathbf{X},\mathbf{T})\right\}=\frac{1}{n}\sum _{i=1}^n{m}_1({\mathbf{X}}_i,{\mathbf{T}}_i)m_2({\mathbf{X}}_i,{\mathbf{T}}_i).\hfill \end{array}$$

The induced norms are denoted as

$${\mid \mid m_1\mid \mid }^2=E\left\{m_1^2(\mathbf{X},\mathbf{T})\right\},{\mid \mid m_1\mid \mid }_n^2=E_n\left\{m_1^2(\mathbf{X},\mathbf{T})\right\}.$$ (A.1)

We now define the polynomial spline approximation space ${\mathcal{M}}_n$,

$${\mathcal{M}}_n=\left\{m_n(\mathbf{x},\mathbf{t})=\sum _{l=1}^{d_1}{g}_l\left(\mathbf{x}\right)t_l;g_l\left(\mathbf{x}\right)=c_{l0}+\sum _{s=1}^{d_2}{g}_{ls}\left(x_s\right);g_{ls}\in {\phi }_s^{0,n}\right\},$$

in which ${\phi }_s^{0,n}=\{g_s:g_s\in {\phi }_s,{\sum }_{i=1}^n{g}_s\left(X_{\text{is}}\right)∕n=0\}$, the empirically centered polynomial spline space. Lemma A.3 shows that the functions in $\mathcal{M}$ and ${\mathcal{M}}_n$ have essentially unique representations.

For any $m\in \mathcal{M}$, let l (m (X, T) , Y) = B [(m (X, T)] Y − C [(m (X, T)]. When they exist, define the log-likelihood and the expected log-likelihood function separately as

$$l_n\left(m\right)=\frac{1}{n}\sum _{i=1}^{n}l(m({\mathbf{X}}_i,{\mathbf{T}}_i),Y_i),\Lambda \left(m\right)=E\left[l(m(\mathbf{X},\mathbf{T}),Y)\right].$$

Note that the expected log-likelihood function Λ (·) needs not to be defined for all $m\in \mathcal{M}$. Therefore we restrict our attention to a subset of $\mathcal{M}$, denoted as ${\mathcal{M}}^{\ast }$, which is a collection of bounded functions in $\mathcal{M}$. That is

$${\mathcal{M}}^{\ast }\{m\in \mathcal{M}:\text{range}\left(m\right)\subset \mathcal{L},\text{for a compact subinterval}\mathcal{L}\subset \mathcal{R}\}.$$

Then Λ (·) is well-defined on ${\mathcal{M}}^{\ast }$, under assumptions (C1)-(C5). The subinterval $\mathcal{L}$ is chosen to be large enough such that η and $2\eta \in {\mathcal{M}}^{\ast }$, where the true predictor function η is bounded on χ₁ × χ₂, under assumptions (C3) and (C6). Similarly, one defines

$${\mathcal{M}}_n^{\ast }\{m_n\in {\mathcal{M}}_n:\text{range}\left(m_n\right)\subset \mathcal{L},\text{for a compact subinterval}\mathcal{L}\subset \mathcal{R}\}.$$

To prove the theoretical results, we need the following assumptions. In what follows, we have denoted by the same letters c, C, any positive constants without distinction in each case. For any function f on χ, denote ||f||_∞ = sup_x∈χ |f (x)| and denote by C^p ([0, 1]), the space of p-times continuously differentiable functions on [0, 1].

• (C1)The function B (·) is twice continuously differentiable and its first derivative B’ (·) is strictly positive.
• (C2)There is a subinterval S of $\mathcal{R}$, such that ρ is concentrated on S, and B” (η) y − C” (η) < 0, for all $\eta \in \mathcal{R}$ and y ∈ S.
• (C3)The tuning variables X = (X₁, . . . , X_d2)^T and the linear covariates T = (T₁, . . . , T_d1)^T are compactly supported and without loss of generality, we assume that their supports are χ₁ = [0, 1]^d₂, and χ₂ = [0, 1]^d₁ respectively.
• (C4)The joint density of X, denoted by f(x), is absolutely continuous and bounded away from zero and infinity, that is, 0 < c ≤ min_x∈χ1 f(x) ≤ max_x∈χ1 f(x) ≤ C < ∞.
• (C5)Let λ₀ (x) ≤ . . . ≤ λ_d1 (x) be the eigenvalues of E(TT^T|X = x). We assume ${\left\{{\lambda }_l\left(\mathrm{x}\right)\right\}}_{l=1}^{d_1}$ are uniformly bounded away from 0, and infinite, for all x ∈ χ₁.
• (C6)The coefficient functions α_ls ∈ C^p+1 ([0, 1]), for l = 1, . . . , d₁, s = 1, . . . , d₂.
• (C7)There is a constant C > 0, such that sup_{x∈χ1,t∈χ2} Var (Y|X = x, T = t) ≤ C.
• (C8)
  The d₂ sets of knots denoted as k_s,n = {0 = x_s,0 ≤ x_s,1 ≤ · · · ≤ x_s,Nn ≤ x_s,Nn+1 = 1}, s = 1, . . . , d₂, are quasi-uniform, that is, there exists c > 0

  $$\underset{s=1,\dots ,d_2}{\max }\frac{\max (x_{s,j+1}-x_{s,j},j=0,\dots ,N_n)}{\min (x_{s,j+1}-x_{s,j},j=0,\dots ,N_n)}\le c.$$

  Furthermore the number of interior knots $N_n\asymp n^{\frac{1}{2p+3}}$, where p denotes the degree of the spline space and ‘≍’ denotes both sides have the same order.

The assumption (C1) implies that the function C (·) is twice continuously differentiable, A (·) is continuously differentiable and A’ (·) is strictly positive. Furthermore, for each $\eta \in \mathcal{R}$, the function B (ξ) A (η) − C (ξ) has a unique maximum at ξ = η. Thus the function that maximizes Λ (·) is given by the true predictor function η. Let h = max_{s=1,...,d2;j=0,...,Nn} |x_s,j+1 − x_s,j|. Then (C8) implies that $h\asymp n^{-\frac{1}{2p+3}}$.

The assumptions (C1)-(C8) are common in polynomial spline estimation literature. Assumptions (C1), (C2) are the same as Assumption 1 and Assumption 2 of Huang (1998), and conditions on Page 591 of Stone (1986). They are satisfied by many familiar exponential families including Normal, Binomial, Poisson, and Gamma distributions. The assumptions (C3)-(C5), (C7) are similar to Assumptions 1-4 in Huang et al. (2002). The assumptions (C6) and (C8) are also used in Xue & Yang (2006b).

A.2 Technical lemmas

The first three lemmas present properties of the spaces $\mathcal{M}$ and ${\mathcal{M}}_n$, which were proved in Xue & Yang (2006b) under more general set-up.

Lemma A.1

Under assumptions (C3)-(C5), there exists a constant c > 0 such that

$${\mid \mid m\mid \mid }_2^2\ge c\left\{\sum _{l=1}^{d_1}\left({\alpha }_{l0}^2+\sum _{s=1}^{d_2}{\mid \mid {\alpha }_{ls}\mid \mid }_2^2\right)\right\}$$

for all $m={\sum }_{l=1}^{d_1}\left({\alpha }_{l0}+{\sum }_{s=1}^{d_2}{\alpha }_{ls}\right)t_l\in \mathcal{M}$.

Lemma A.2

Under assumptions (C3)-(C8), one has

$$\underset{g_1\in {\mathcal{M}}_n,g_2\in {\mathcal{M}}_n}{\sup }\mid \frac{{\langle g_1,g_2\rangle }_n-\langle g_1,g_2\rangle }{\mid \mid g_1\mid \mid \mid \mid g_2\mid \mid }\mid =O_p\left(\sqrt{\frac{{\log }^2\left(n\right)}{nh}}\right).$$

In particular, there exists constants 0 < c < 1 < C, such that except on an event whose probability tends to zero, as $n\to \infty ,c\parallel g\parallel \le {\parallel g\parallel }_n\le C\parallel g\parallel ,\forall g\in {\mathcal{M}}_n$.

Lemma A.3

Under assumptions (C3)-(C8), the model space $\mathcal{M}$ is theoretically identifiable, i.e., for any $m\in \mathcal{M},\parallel m\parallel =0$, implies m = 0 a.s.. The approximation space ${\mathcal{M}}_n$ is empirically identifiable, for any $m_n\in {\mathcal{M}}_n,{\parallel m_n\parallel }_n=0$, implies m_n = 0 a.s..

Lemma A.4

Under assumptions (C1)-(C6), Λ(·) is strictly concave over ${\mathcal{M}}^{\ast }$. That is, for any $m_0,m_1\in {\mathcal{M}}^{\ast }$, that are essentially different (different on a set of positive probability relative to the joint distribution of (X, T)),

$$\Lambda (m_0+t(m_1-m_0))>(1-t)\Lambda \left(m_0\right)+t\Lambda \left(m_1\right).$$

Proof: For any $m\in {\mathcal{M}}^{\ast }$, one has

$$\Lambda \left(m\right)=E\{B\left[m(\mathbf{X},\mathbf{T})\right]Y-C\left[m(\mathbf{X},\mathbf{T})\right]\}=E\{B\left[m\left((\mathbf{X},\mathbf{T})\right)\right]A\left[\eta (\mathbf{X},\mathbf{T})\right]-C\left[m(\mathbf{X},\mathbf{T})\right]\}.$$

Note that A [η (X, Y)] ∈ S almost surely, since ρ is concentrated on S (Assumption C2). Therefore, Assumption C2 gives B” [m (X, T)] A [η (X, Y)] − C” [m (X, T)] < 0, almost surely, for any $m\in {\mathcal{M}}^{\ast }$. Let λ [m, X, T] = B [m (X, T)] A [η (X, Y)] − C [m (X, T)] and m^(t) = m₀ + t (m₁ − m₀). As a result, almost surely,

$$\lambda [m^{\left(t\right)},\mathbf{X},\mathbf{T}]>t\lambda [m_0,\mathbf{X},\mathbf{T}]+(1-t)\lambda [m_1,\mathbf{X},\mathbf{T}].$$

The lemma follows from above inequality.

Lemma A.5

Under assumptions (C1)-(C6), there exist positive numbers c₁ and c₂, such that for all $m\in {\mathcal{M}}^{\ast }$,

$$-c_1{\mid \mid m-n\mid \mid }^2\le \Lambda \left(m\right)-\Lambda \left(\eta \right)\le -c_2{\mid \mid m-n\mid \mid }^2.$$

Proof: For any $m\in {\mathcal{M}}^{\ast }$. Let m^(t) = (1 − t) η + tm. Then

$$\frac{d}{dt}\Lambda \left(m^{\left(t\right)}\right){\mid }_{t=0}=0,$$

since η is the true predictor function. Hence integration by parts gives

$$\Lambda \left(m\right)-\Lambda \left(\eta \right)={\int }_0^1(1-t)\frac{d^2}{d{t}^2}\Lambda \left(m^{\left(t\right)}\right)dt,$$

where

$$\frac{d^2}{d{t}^2}\Lambda \left(m^{\left(t\right)}\right)=E\left\{{(m(\mathbf{X},\mathbf{T})-\eta (\mathbf{X},\mathbf{T}))}^2[B^{\prime \prime }\left(m^{\left(t\right)}(\mathbf{X},\mathbf{T})\right)A\left(\eta (\mathbf{X},\mathbf{T})\right)-C^{\prime \prime }\left(m^{\left(t\right)}(\mathbf{X},\mathbf{T})\right)]\right\}.$$

Assumptions (C1) & (C2) entail that there exist constants $c_1^{\ast },c_2^{\ast }>0$ such that,

$$\begin{array}{c}\hfill {\inf }_{m\in {\mathcal{M}}^{\ast }}[B^{\prime \prime }\left(m\right)A\left(\eta \right)-C^{\prime \prime }\left(m^{\left(t\right)}\right)]\ge -c_2^{\ast }\hfill \\ \hfill {\sup }_{m\in {\mathcal{M}}^{\ast }}[B^{\prime \prime }\left(m\right)A\left(\eta \right)-C^{\prime \prime }\left(m^{\left(t\right)}\right)]\le -c_1^{\ast }\hfill \end{array}$$

The proof of Lemma is complected by taking $c_1=c_1^{\ast }∕2$ and $c_2=c_2^{\ast }∕2$. The proof is similar to those in the Lemma 4.1 of Huang (1998) and the Lemma 6 of Stone (1986).

A.3 Proof of Theorem 1

Note that $\eta ={\mathrm{argmax}}_{m\in {\mathcal{M}}^{\ast }}\Lambda \left(m\right)$, and $\widehat{\eta }={\mathrm{argmax}}_{m\in {\mathcal{M}}_n}{l}_n\left(m\right)$. Write ${\eta }^{\ast }={\mathrm{argmax}}_{m\in {\mathcal{M}}_n}\Lambda \left(m\right)$. Then one has the following error decomposition,

$$\widehat{\eta }-\eta =\widehat{\eta }-{\eta }^{\ast }+{\eta }^{\ast }-\eta ,$$

where $\widehat{\eta }-{\eta }^{\ast }$ and η* − η can be understood as the estimation error and the approximation error respectively. Then the first part of Theorem 1, $\parallel \widehat{\eta }-\eta \parallel =O_p(h^{p+1}+1∕\sqrt{nh})$ is obtained by showing that ||η* - η|| = O_p(h^p+1), and $\parallel \widehat{\eta }-{\eta }^{\ast }\parallel =O_p(1∕\sqrt{nh})$, which is proved in Step1 and Step 2 respectively. The rest of Theorem 1 follows immediately from Lemma A.1. The same idea of error decomposition is also used in Huang (1998) where the generalized functional ANOVA model is assumed instead.

Step1: (the approximation error)

Lemma A.5 entails that there exist constants c₁,c₂ > 0, such that for all $m\in {\mathcal{M}}^{\ast }$,

$$-c_1{\mid \mid m-n\mid \mid }^2\le \Lambda \left(m\right)-\Lambda \left(\eta \right)\le -c_2{\mid \mid m-n\mid \mid }^2.$$ (A.2)

Thus, for any constant c > 0, and any $m_n\in {\mathcal{M}}_n^{\ast }\subset {\mathcal{M}}^{\ast }$ with ||m_n − η|| = ch^p+1 (if such m_n exists), (A.2) entails that,

$$\Lambda \left(m_n\right)-\Lambda \left(\eta \right)\le -c_2{c}^2{h}^{2(p+1)}.$$ (A.3)

On the other hand, the approximation theorem (de Boor, 2001) ensures that, there exist spline functions $g_{ls}\in {\phi }_s^0$, and a constant C > 0 that does not depend on n, such that ||g_ls − α_ls||_∞ ≤ Ch^p+1, for 1 ≤ 1 ≤ d₁, 1 ≤ s ≤ d₂. Let $m_n^{\ast }={\sum }_{l=1}^{d_1}\left\{{\alpha }_{l0}+{\sum }_{s=1}^{d_2}{g}_{ls}\left(x_s\right)\right\}{t}_l\in {\mathcal{M}}_n^{\ast }$. By assumption (C3), one has

$$\mid \mid m_n^{\ast }-n\mid \mid \le \sum _{l=1}^{d_1}\sum _{s=1}^{d_2}\mid \mid (g_{ls}-{\alpha }_{ls})t_l\mid \mid \le \sum _{l=1}^{d_1}\sum _{s=1}^{d_2}{\mid \mid g_{ls}-{\alpha }_{ls}\mid \mid }_{\infty }\le c_3{h}^{p+1},$$

in which c₃ = d₁d₂C. Thus (A.2) also gives

$$\Lambda \left(m_n^{\ast }\right)-\Lambda \left(\eta \right)\ge -c_1{c}_3^2{h}^{2(p+1)}.$$ (A.4)

By choosing c such that $c>c_3\sqrt{c_1∕c_2}$, (A.3) and (A.4) entail that, when n is sufficiently large,

$$\Lambda \left(m_n\right)<\Lambda \left(m_n^{\ast }\right),\text{for any}{m}_n\in S\left(c\right),$$ (A.5)

where $S\left(c\right)=\{m_n:m_n\in {\mathcal{M}}_n^{\ast },\text{with}\parallel m_n-\eta \parallel =c{h}^{p+1}\}$. Note that for such choice of $c,\parallel m_n^{\ast }-\eta \parallel \le c_3{h}^{p+1}<c{h}^{p+1}$. Let $\{m_n:m_n\in {\mathcal{M}}_n^{\ast },\text{with}\parallel m_n-\eta \parallel \le c{h}^{p+1}\}$, which is a closed bounded convex set in ${\mathcal{M}}_n^{\ast }$. Assumption (C1) and Lemma A.5 entail that Λ (·) is a continuous concave functional on ${\mathcal{M}}_n^{\ast }$. Therefore, Theorem 2 in Pietrzykowski (1972) ensures that Λ (·) has a maximum on B(c). On the other hand, (A.5) ensures that the maximum must be in the interior of B(c). Together with the concavity of Λ (·) and the definition of η*, η* exists, and satisfies ||η* − η|| < ch^p+1, for n sufficiently large. Hence ||η* − η|| = O_p(h^p+1).

Step2 (the estimation error)

Let $\varphi ={\left\{{\varphi }_j\right\}}_{j=1}^{I_n}$ be an orthornormal basis for ${\mathcal{M}}_n$ with respect to the theoretical inner product, where I_n = (1 + d₂J_n) d₁. Then one can write $\widehat{\eta }={\sum }_{j=1}^{I_n}{\widehat{\beta }}_j{\varphi }_j$, and ${\eta }^{\ast }={\sum }_{j=1}^{I_n}{\beta }_j^{\ast }{\varphi }_j$, for some coefficients $\widehat{\beta }={({\widehat{\beta }}_1,\dots ,{\widehat{\beta }}_{I_n})}^T$, and ${\beta }^{\ast }={({\beta }_1^{\ast },\dots ,{\beta }_{I_n}^{\ast })}^T$. For any $m\in {\mathcal{M}}_n$, write $l_n\left(m\right)=l_n\left({\sum }_{j=1}^{I_n}{\beta }_j{\varphi }_j\right)$, as l_n (β). Let S (β) = (∂/∂β) l_n (β) be the score at β, which is a I_n-dimensional vector having entries

$$\frac{\partial }{\partial {\beta }_j}{l}_n\left(\beta \right)=\frac{1}{n}\sum _{i=1}^n{\varphi }_j({\mathbf{X}}_i,{\mathbf{T}}_i)\{B^{\prime }\left[m({\mathbf{X}}_i,{\mathbf{T}}_i)\right]Y_i-C^{\prime }\left[m({\mathbf{X}}_i,{\mathbf{T}}_i)\right]\},$$

and let D (β) = (∂²/∂β∂β^T) l_n (β) I_n × I_n Hessian matrix, which has entries

$$\frac{\partial }{\partial {\beta }_j\partial {\beta }_{j^{\prime }}}{l}_n\left(\beta \right)=\frac{1}{n}\sum _{i=1}^n{\varphi }_j({\mathbf{X}}_i,{\mathbf{T}}_i){\varphi }_{j^{\prime }}({\mathbf{X}}_i,{\mathbf{T}}_i)\{B^{\prime \prime }\left[m({\mathbf{X}}_i,{\mathbf{T}}_i)\right]Y_i-C^{\prime \prime }\left[m({\mathbf{X}}_i,{\mathbf{T}}_i)\right]\}.$$

Then, for any $m={\sum }_{j=1}^{I_n}{\beta }_j{\varphi }_j\in {\mathcal{M}}_n$ with ${\left\{{\varphi }_j\right\}}_{j=1}^{I_n}$ being a set of orthornormal basis and $\parallel m-{\eta }^{\ast }\parallel =\mid \beta -{\beta }^{\ast }\mid =a∕\sqrt{nh}$, for some constant a > 0 to be chosen later, Taylor expansion gives

$$\begin{array}{cc}\hfill l_n\left(\beta \right)=l_n& \left({\beta }^{\ast }\right)+{(\beta -{\beta }^{\ast })}^T\mathbf{S}\left({\beta }^{\ast }\right)\hfill \\ \hfill & +{(\beta -{\beta }^{\ast })}^T\left[{\int }_0^1(1-t)\mathbf{D}({\beta }^{\ast }+t(\beta -{\beta }^{\ast }))dt\right](\beta -{\beta }^{\ast }).\hfill \end{array}$$

For any fixed ε > 0, Lemma A.7 implies that one can choose a sufficiently large, such that $P(\mid \mathbf{S}\left({\beta }^{\ast }\right)\mid <a{c}_5∕\sqrt{nh})\ge 1-\epsilon$. Let $\mathcal{A}=\{\mid \mathbf{S}\left({\beta }^{\ast }\right)\mid <a{c}_5∕\sqrt{nh}\}$. Then on event $\mathcal{A}$

$$\mid {(\beta -{\beta }^{\ast })}^T\mathbf{S}\left({\beta }^{\ast }\right)\mid \le \mid \beta -{\beta }^{\ast }\mid \mid \mathbf{S}\left({\beta }^{\ast }\right)\mid <c_5{a}^2∕\left(nh\right).$$ (A.6)

Moreover, for such a, Lemma A.8 implies that

$$\begin{array}{cc}\hfill {(\beta -{\beta }^{\ast })}^T\left[{\int }_0^1(1-t)\mathbf{D}({\beta }^{\ast }+t(\beta -{\beta }^{\ast }))dt\right](\beta -{\beta }^{\ast })\le & -c_5{\mid \beta -{\beta }^{\ast }\mid }^2\hfill \\ \hfill & =-c_5{a}^2∕\left(nh\right)\hfill \end{array}$$ (A.7)

except on an event whose probability tends to zero, as n → ∞ Thus (A.6) & (A.7) entails that, except on an event whose probability tends to zero, as n → ∞, l_n (β) < l_n (β*) for all $\mid \beta -{\beta }^{\ast }\mid =a∕\sqrt{nh}$. Hence by the concavity of l_n (β) (Lemma A.6) and similar arguments as in Step 1, $\widehat{\eta }$ exists and satisfies $\mid \mid \widehat{\eta }-{\eta }^{\ast }\mid \mid <a∕\sqrt{nh}$. Since ε is arbitrary, $\widehat{\eta }$ exists except on an event whose probability tends to zero as n → ∞, and satisfies $\mid \mid \widehat{\eta }-{\eta }^{\ast }\mid \mid =O_p(1∕\sqrt{nh})$.

In the following, we present the necessary Lemmas used in the proof. The Lemmas are presented here because they need notations introduced in the proof of Theorem 1.

Lemma A.6

Under assumptions (C1)-(C8), there exits a c₄ > 0, such that, except on an event whose probability tends to zero as n → ∞

$$\frac{d^2}{d{t}^2}{l}_n[m_1+t(m_2-m_1)]\le -c_4{\mid \mid m_2-m_1\mid \mid }^2,$$

for 0 < t < 1 and all $m_1,m_2\in {\mathcal{M}}_n^{\ast }$. Thus the log-likelihood l_n (·) is strictly concave on ${\mathcal{M}}_n^{\ast }$ except on an event whose probability tends to zero as n → ∞.

Proof: Let m_t = m₁ + t (m₂ − m₁), for 0 < t < 1. It follows from Assumptions (C1) and (C2) that

$$\begin{array}{cc}\hfill \frac{d^2}{d{t}^2}{l}_n\left(m_t\right)=\frac{1}{n}\sum _{i=1}^n& \{{[m_2({\mathbf{X}}_i,{\mathbf{T}}_i)-m_1({\mathbf{X}}_i,{\mathbf{T}}_i)]}^2\hfill \\ \hfill & [B^{\prime \prime }\left[m_t({\mathbf{X}}_i,{\mathbf{T}}_i)\right]Y_i-C^{\prime \prime }\left[m_t({\mathbf{X}}_i,{\mathbf{T}}_i)\right]]\},\hfill \end{array}$$

Note that there is a constant δ > 0, such that B” (ξ) y − C” (ξ) ≤ −δ for all $\xi \in \mathcal{L}$ y ∈ S. Thus the right-hand side of the above equality is bounded by

$$-\frac{\delta }{n}\sum _{i=1}^n{[m_2({\mathbf{X}}_i,{\mathbf{T}}_i)-m_1({\mathbf{X}}_i,{\mathbf{T}}_i)]}^2=-\delta {\mid \mid m_2-m_1\mid \mid }_n^2\le -c\delta {\mid \mid m_2-m_1\mid \mid }^2,$$

by Lemma A.2. The result follows by letting c₄ = cδ.

Lemma A.7

Under assumptions (C1)-(C8), for any constant c > 0,

$$\underset{a\to \infty }{\lim }\underset{n\to \infty }{\lim \sup }P(\mid \mathbf{S}\left({\beta }^{\ast }\right)\mid \ge ca∕\sqrt{nh})=0$$

Proof: Note that β* maximizes

$$\Lambda \left(\beta \right)=\Lambda \left(\sum _{j=1}^{I_n}{\beta }_j{\varphi }_j\right)=E\left[B\left(\sum _{j=1}^{I_n}{\beta }_j{\varphi }_j(\mathbf{X},\mathbf{T})\right)Y-C\left(\sum _{j=1}^{I_n}{\beta }_j{\varphi }_j(\mathbf{X},\mathbf{T})\right)\right].$$

Thus ${\frac{\partial }{\partial \beta }\Lambda \left(\beta \right)\mid }_{\beta ={\beta }^{\ast }}=0$, which implies

$$E\left\{{\varphi }_j(\mathbf{X},\mathbf{T})[B^{\prime }\left({\eta }^{\ast }\right)Y-C^{\prime }\left({\eta }^{\ast }\right)]\right\}=0,\text{for}1\le j\le I_n.$$

Thus

$$E(\mid \mathbf{S}{({\beta }^{\ast }\mid }^2)=\sum _{j=1}^{I_n}E{\left[\frac{\partial }{\partial {\beta }_j}{l}_n\left({\beta }^{\ast }\right)\right]}^2=\frac{1}{n}\sum _{j=1}^{I_n}\mathrm{Var}\left\{{\varphi }_j(\mathbf{X},\mathbf{T})[B^{\prime }\left({\eta }^{\ast }\right)Y-C^{\prime }\left({\eta }^{\ast }\right)]\right\},$$

where

$$\begin{array}{cc}\hfill \mathrm{Var}& \left\{{\varphi }_j(\mathbf{X},\mathbf{T})[B^{\prime }\left({\eta }^{\ast }\right)Y-C^{\prime }\left({\eta }^{\ast }\right)]\right\}\hfill \\ \hfill & =E\left[\mathrm{Var}\{{\varphi }_j(\mathbf{X},\mathbf{T})[B^{\prime }\left({\eta }^{\ast }\right)Y-C^{\prime }\left({\eta }^{\ast }\right)]\mid \mathbf{X},\mathbf{T}\}\right]\hfill \\ \hfill & +\mathrm{Var}\left[E\{{\varphi }_j(\mathbf{X},\mathbf{T})[B^{\prime }\left({\eta }^{\ast }\right)Y-C^{\prime }\left({\eta }^{\ast }\right)]\mid \mathbf{X},\mathbf{T}\}\right]\hfill \\ \hfill & \le E\left[{\varphi }_j^2(\mathbf{X},\mathbf{T}){\left(B^{\prime }\left({\eta }^{\ast }\right)\right)}^2{\sigma }^2(\mathbf{X},\mathbf{T})\right]\le cE\left[{\varphi }_j^2(\mathbf{X},\mathbf{T})\right]=C{\mid \mid {\varphi }_j\mid \mid }^2.\hfill \end{array}$$

Thus

$$E(\mid \mathbf{S}({\beta }^{\ast }{\mid }^2)\le \frac{c}{n}\sum _{j=1}^{I_n}E\left[{\varphi }_j^2(\mathbf{X},\mathbf{T})\right]=\frac{c}{n}\sum _{j=1}^{I_n}{\mid \mid {\varphi }_j\mid \mid }^2=\frac{c[1+d_2(N_n+p)]d_1}{n}\asymp \frac{1}{nh},$$

and we complete the proof of Lemma.

Lemma A.8

Under assumptions (C1)-(C8), there exists a constant c₅ > 0, such that, for any fixed positive constant a, with probability approaching to 1, as n → ∞,

$${(\beta -{\beta }^{\ast })}^T\left[{\int }_0^1(1-t)\mathbf{D}({\beta }^{\ast }+t(\beta -{\beta }^{\ast }))dt\right](\beta -{\beta }^{\ast })\le -c_5{\mid \beta -{\beta }^{\ast }\mid }^2$$

for all $\beta \in {\mathcal{R}}^n$, with $\mid \beta -{\beta }^{\ast }\mid =a∕\sqrt{nh}$.

Proof: For any $m\in {\mathcal{M}}_n$ with $\mid \mid m-n^{\ast }\mid \mid =a∕\sqrt{nh}$, Lemma A.6 entails that there exits a constant c₄ > 0, such that, for 0 < t < 1,

$$\frac{d^2}{d{t}^2}{l}_n({\eta }^{\ast }+t(m-{\eta }^{\ast }))\le -c_4{\mid \mid m-n^{\ast }\mid \mid }^2,$$

except on an event whose probability tends to zero as n → ∞. Also note that

$$\begin{array}{cc}\hfill \frac{d^2}{d{t}^2}{l}_n({\eta }^{\ast }+t(m-{\eta }^{\ast }))=& \frac{d^2}{d{t}^2}{l}_n({\beta }^{\ast }+t(\beta -{\beta }^{\ast }))\hfill \\ \hfill =& {(\beta -{\beta }^{\ast })}^T\mathbf{D}({\beta }^{\ast }+t(\beta -{\beta }^{\ast }))(\beta -{\beta }^{\ast }).\hfill \end{array}$$

Thus the Lemma follows with $c_5=c_4∕{\int }_0^1(1-t)dt=c_4∕2$.

A.4 Proof of Theorem 2

We prove the result only when the null hypothesis H₀ : α_ls = 0, for notation convenience. For higher order polynomials, the proof follows similarly. Define the approximation space under H₀ as

$$\begin{array}{cc}\hfill {\mathcal{M}}_n^0=\{m_n(\mathbf{x},\mathbf{t})\phantom{\}}& =\sum _{l^{\prime }\ne l,l^{\prime }=1}^{d_1}\left[{\alpha }_{l^{\prime }0}+\sum _{s^{\prime }=1}^{d_2}{g}_{l^{\prime }{s}^{\prime }}\left(x_{s^{\prime }}\right)\right]t_{l^{\prime }}\hfill \\ \hfill & +\phantom{\{}\left[{\alpha }_{l0}+\sum _{s^{\prime }\ne s,s^{\prime }=1}^{d_2}{g}_{l{s}^{\prime }}\left(x_{s^{\prime }}\right)\right]t_l;g_{ls}\in {\phi }_s^{0,n}\},\hfill \end{array}$$

which leaves out the spline approximation term for α_ls. Note that ${\mathcal{M}}_n^0\subset {\mathcal{M}}_n$. Denote ${\widehat{\eta }}^0={\mathrm{argmax}}_{m\in {\mathcal{M}}_n^0}{l}_n\left(m\right)$, the restricted maximum likelihood estimator of η under H₀. Recall that $\widehat{\eta }={\mathrm{argmax}}_{m\in {\mathcal{M}}_n}{l}_n\left(m\right)$, the unrestricted MLE. Write ${\widehat{\eta }}^0={\sum }_{j=1}^{I_n}{\widehat{\beta }}_j^0{\varphi }_j$, and $\widehat{\eta }={\sum }_{j=1}^{I_n}{\widehat{\beta }}_j{\varphi }_j$, for some coefficients ${\widehat{\beta }}^0={({\beta }_1^0,\dots ,{\widehat{\beta }}_{I_n}^0)}^T$, and $\widehat{\beta }={({\beta }_1,\dots ,{\widehat{\beta }}_{I_n})}^T$. Then Taylor expansion gives

$$\begin{array}{cc}\hfill l_n\left({\widehat{\eta }}^0\right)-l_n& \left(\widehat{\eta }\right)=l_n\left({\widehat{\beta }}^0\right)-l_n\left(\widehat{\beta }\right)={({\widehat{\beta }}^0-\widehat{\beta })}^T\mathbf{S}\left(\widehat{\beta }\right)\hfill \\ \hfill & +{({\widehat{\beta }}^0-\widehat{\beta })}^T\left[{\int }_0^1(1-t)\mathbf{D}(\widehat{\beta }+t({\widehat{\beta }}^0-\widehat{\beta }))dt\right]({\widehat{\beta }}^0-\widehat{\beta }),\hfill \end{array}$$

in which S (β̂) = 0 by the definition of MLE, and

$$\begin{array}{cc}\hfill & {({\widehat{\beta }}^0-\widehat{\beta })}^T\left[{\int }_0^1(1-t)\mathbf{D}(\widehat{\beta }+t({\widehat{\beta }}^0-\widehat{\beta }))dt\right]({\widehat{\beta }}^0-\widehat{\beta })=\hfill \\ \hfill & \frac{1}{n}\sum _{i=1}^n\{{[{\widehat{\eta }}^0({\mathbf{X}}_i,{\mathbf{T}}_i)-\widehat{\eta }({\mathbf{X}}_i,{\mathbf{T}}_i)]}^2\phantom{\}}\hfill \\ \hfill & \phantom{\{}{\int }_0^1(1-t)[B^{\prime \prime }\left({\eta }_t({\mathbf{X}}_i,{\mathbf{T}}_i)\right)Y_i-C^{\prime \prime }\left({\eta }_t({\mathbf{X}}_i,{\mathbf{T}}_i)\right)]dt\},\hfill \end{array}$$

where η_t = η̂ + t (η̂⁰ − η̂). By (C1) and (C2), there exists constants 0 < c < C, such that −C ≤ B” (ξ) y − C” (ξ) ≤ −c, for all $\xi \in \mathcal{L}$, and y ∈ R. Also note that ${\int }_0^1(1-t)dt=1∕2$, and $\frac{1}{n}{\sum }_{i=1}^n{\{{\widehat{\eta }}^0({\mathbf{X}}_i,{\mathbf{T}}_i)-\widehat{\eta }({\mathbf{X}}_i,{\mathbf{T}}_i)\}}^2={\mid \mid {\widehat{\eta }}^0-\widehat{\eta }\mid \mid }_n^2$. Therefore T_n = 2[l_n (η̂) − l_n (η̂⁰)] satisfies $c{\mid \mid {\widehat{\eta }}^0-\widehat{\eta }\mid \mid }_n^2\le T_n\le C{\mid \mid {\widehat{\eta }}^0-\widehat{\eta }\mid \mid }_n^2$. That is $T_n\asymp {\mid \mid {\widehat{\eta }}^0-\widehat{\eta }\mid \mid }_n^2$. Lemma A.2 further gives that T_n ≍ ||η̂⁰ − η̂||². Then the result of Theorem 2 follows by noting the following. Under H₀, ||η̂⁰ − η̂|| ≤ ||η̂⁰ − η|| + ||η̂ − η|| = o_p (1). while under H₁, ||η̂⁰ − η̂|| ≥ ||η̂⁰ − η|| − ||η̂ − η||, in which by Lemma A.1, ||η̂⁰ − η||² ≥ c ||α_ls||², and ||η̂ − η|| = o_p (1).