Lack-of-fit Tests for Generalized Linear Models via Splines

Abstract

Cubic B-splines are used to estimate the nonparametric component of a semiparametric generalized linear model. A penalized log-likelihood ratio test statistic is constructed for the null hypothesis of the linearity of the non-parametric function. When the number of knots is fixed, its limiting null distribution is the distribution of a linear combination of independent chi-squared random variables, each with one df. The smoothing parameter is determined by giving a specified value for its asymptotically expected value under the null hypothesis. A simulation study is conducted to evaluate its power performance; a real-life dataset is used to illustrate its practical use.

1. Introduction

Generalized linear models (GLM) offer a unifying framework of regression analysis for data from the exponential family of distributions, which include Gaussian, Poisson, and binomial distributions among others (Nelder and Wedderburn, 1972; McCullagh and Nelder, 1989). More specifically, in a GLM, the response variable Y is assumed to have a density (or probability mass) function of the following form

$$f(y;\theta ,\phi )=exp\left(\frac{y\theta -b(\theta )}{a(\phi )}+c(y,\phi )\right),$$ (1)

where φ is a scale or nuisance parameter; θ is the natural parameter of interest; a(φ), b(θ), and c(y, φ) are functions of known form, which determine a specific member in the one-parameter exponential family of distributions. It can be shown that the mean response μ = E(Y) = b′(θ) and the variance of the response var(Y ) = a(φ)b″(θ) = a(φ)v(μ), where v(μ) = b″(θ) is a variance function depending only on μ. Let u be the covariate of interest and x = (x₀, x₁, …, x_p−1) the other covariates in the model, where x₀ = 1. Under this formulation, we model the mean response μ = E(Y) = b′(θ), related to the covariate vector z = (x, u), through a monotonic differentiable link function g as follows:

$$g(\mu )=g[b^{\prime }(\theta )]=\mathit{x}\mathit{\alpha }+s(u).$$ (2)

Here α = (α₀, …, α_p−1)^T is a vector of unknown p model parameters associated with the covariate vector x. s is an unknown but smooth function of u, which is to be estimated from the data, while the effects of x remain linear. The model in (2) is a generalized partially linear model and referred to as a semiparametric GLM, because it contains both parametric and nonparametric components.

There have been several methods of estimation proposed for estimating the parameters in (2), including, e.g., splines (Green and Yandell, 1985; Green, 1987; Hastie and Tibshirani, 1990; Green and Silverman, 1994; Ruppert, Wand, and Carroll, 2003), local likelihood (Kauermann and Tutz, 2001), and weighted quasi-likelihood (Severini and Staniswalis, 1994). Non-parametric regression techniques have been recently used for testing lack of fit of parametric regression models; see, e.g., Aerts, Claeskens, and Hart (1999), Cantoni and Hastie (2002), Crainiceanu and Ruppert (2004), Crainiceanu et al. (2005), and Liu and Wang (2004). Additionally, Hart (1997) gave a detailed survey on lack-of-fit tests using nonparametric regression techniques.

In contrast to these methods, we use a flexible alternative. That is, splines with a moderate number of knots are used. Specifically, the unknown functional form of the effect of the covariate u, s(u), is modeled by a linear combination of fixed-knot cubic B-spline basis functions (Schoenberg, 1946; Curry and Schoenberg, 1966). The model fitting is achieved by maximizing the penalized log-likelihood. A second-order difference penalty on the adjacent B-spline coefficients is imposed in order to prevent overfitting while estimating model parameters. In addition to the added modeling flexibility, the proposed model can be used to test for the linear effect of u.

The paper is organized as follows. In Section 2, we introduce the structure of the semiparametric GLM and the method of estimation of the model parameters. A penalized log-likelihood ratio test is constructed for the linear effect of the covariate u. Section 3 presents simulation results to study the power performance of the proposed test. In Section 4, we illustrate the use of the proposed method with a real-life example. Some discussion is given in Section 5. Proofs and technical conditions are given in the Appendix.

2. Method

2.1 Estimation

The unknown smooth function s can be approximated by a variety of spline functions, but the s is approximated by using a fixed-knot cubic spline because it provides the best compromise between computational cost and smoothness. The plus-function terms are intuitive and, hence, the truncated power basis is attractive from a statistical point of view (Smith, 1979). However, the B-spline basis provides a numerically superior alternative basis to the truncated power basis. Therefore, we model s as a linear combination of cubic B-spline basis functions with q preselected knots. We choose the kth knot corresponding to the $\frac{k}{q+1}\text{th}$ sample quantile of the distinct values of u_is. Let B₁(u), …, B_q+4(u) be the cubic B-spline basis functions. See de Boor (2001) for details of computation of B-splines and their mathematical properties. We are interested in testing the hypothesis that the effect of u is linear, i.e., testing H₀: s(u) = τu versus H_a: s(u) ≠ τu, where τ is an unknown parameter. The space of cubic B-splines includes the constant and linear functions and the constant is given in the parametric component of the model in (2). Therefore, to model s and easily specify the null hypothesis of the linear effect of the covariate u, the linear term is given separately in (3). Consequently, it is needed to drop any two of the q + 4 cubic B-spline basis functions to have the full-rank model

$$s(u)=\tau u+\sum _{k=1}^K{\beta }_k{B}_k(u).$$ (3)

Here K = q + 2. The β_ks are cubic B-spline coefficients.

Let β = (β₁, …, β_K)^T and B_u = (B₁(u), …, B_K(u)). By writing s(u) = τu + B_uβ, we express the mean response μ = b′(θ) in (2) as g(μ) = g[b′(θ)] = xα + τu + B_uβ, which can be written as in vector notation

$$g(\mu )=g[b^{\prime }(\theta )]=\mathit{z}\mathit{\xi }+{\mathit{B}}_u\mathit{\beta }={\mathit{A}}_{\mathit{z}}\mathit{\psi },$$ (4)

where A_z = (z, B_u) = (x, u, B_u) and ψ = (ξ^T, β^T)^T for ξ = (α^T, τ)^T. Let Y_i be the response from the ith subject with the covariate vector z_i = (x_i, u_i), and Y_i follows the one-parameter exponential family of distributions as described by (1) with mean μ_i = b′(θ_i), i = 1, 2, …, n. In most applications, the so-called canonical link function is used, i.e., g(·) = (b′)⁻¹(·). For example, the model in (2) is a semiparametric Poisson regression model if g(·) is a logarithmic function; a semiparametric logistic regression model if g(·) is a logit function. Therefore, we focus on the case of canonical link function. The model in (4) can be expressed as θ_i = g[b′(θ_i)] = A_ziψ.

By plugging θ_i = A_ziψ into f(y_i; θ_i, φ), i = 1, …, n, we can obtain the log-likelihood function $\ell (\mathit{\psi },\phi )={\sum }_{i=1}^n[y_i{\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }-b({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi })]/a(\phi )+{\sum }_{i=1}^{n}c(y_i,\phi )$. The scale parameter φ does not affect the estimating equation that leads to the estimate of ψ and a(φ) may be estimated consistently. Thus, for simplicity and without loss of generality, we take a(φ) = 1. Additionally, ${\sum }_{i=1}^{n}c(y_i,\phi )$ does not depend on ψ. Therefore, the log-likelihood function for the model in (1) with s(u) = τu + B_uβ is $\ell (\mathit{\psi })={\sum }_{i=1}^n[y_i{\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }-b({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi })]$. For example, if Y_i ~ Poisson(μ_i), $\ell (\mathit{\psi })={\sum }_{i=1}^n[y_i{\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }-exp({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi })]$. $\ell (\mathit{\psi })={\sum }_{i=1}^n[y_i{\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }-log(1+exp({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }))]\text{if}{Y}_i~\text{Bernoulli}({\mu }_i)$. Similarly, one can derive the log-likelihood functions for other members of the one-parameter exponential distribution family.

To estimate ψ and overcome the problem of overfitting the data, we employ the penalized likelihood technique introduced by Good and Gaskins (1971) in the context of nonparametric probability density estimation. Thus, by adding the roughness penalty −(1/2)β^T β to the ℓ(ψ) we obtain the penalized log-likelihood ℓ_p(ψ; λ) = ℓ(ψ) −(1/2)λβ^T β. Here = is a K × K matrix, where is a matrix representation of the difference operator Δ². ${\mathit{\beta }}^{\text{T}}\mathcal{G}\mathit{\beta }={\sum }_{k=3}^K{({\mathrm{\Delta }}^2{\beta }_k)}^2$, which is a discrete approximation to ∫s″(t)²dt, where Δ²β_k = β_k − 2β_k−1 + β_k−2; see Eilers and Marx (1996). The smoothing parameter λ > 0 controls the tradeoff between fidelity to the data and smoothness of the fitting. Let be a (p + 1 + K) × (p + 1 + K) matrix with zeros in the first p + 1 rows and columns and in the remainder of the matrix. The ℓ_p(ψ; λ) can then be rewritten as

$${\ell }_p(\mathit{\psi };\lambda )=\ell (\mathit{\psi })-\frac{1}{2}\lambda {\mathit{\psi }}^{\text{T}}\mathcal{Q}\mathit{\psi }.$$ (5)

For given λ > 0, let ψ̂_λ = (ξ̂_λ, β̂_λ) be the value of ψ maximizing the ℓ_p(ψ; λ), which is the maximum penalized likelihood estimate (MPLE) of ψ and can be obtained by solving the following weighted least-squares equations (A^TWA + λ )ψ = A^TWỹ. Here $\mathit{A}=[\begin{array}{ll}\mathit{Z}\hfill & \mathit{B}\hfill \end{array}]$ is an n×(p+1+K) matrix for $\mathit{Z}={({\mathit{z}}_1^{\text{T}},\dots ,{\mathit{z}}_n^{\text{T}})}^{\text{T}}$ being an n×(p+1) matrix and $\mathit{B}={({\mathit{B}}_{u_1}^{\text{T}},\dots ,{\mathit{B}}_{u_n}^{\text{T}})}^{\text{T}}$ an n × K matrix. W = diag(w₁, …, w_n) is an n × n matrix for w_i = b″(A_ziψ). ỹ = (ỹ₁, …, ỹ_n)^T is a working response vector for ỹ_i = A_ziψ + [y_i − b′(A_ziψ)]/b″(A_ziψ).

2.2 A penalized log-likelihood ratio test

It turns out from the model in (3) that testing H₀: s(u) = τu is equivalent to testing the hypothesis that β = 0. It is noted that under H₀: β = 0, ψ = 0 and, hence, ℓ_p(ψ; λ) = ℓ(ψ). Let ${\mathit{\xi }}_0={({\mathit{\alpha }}_0^{\text{T}},{\tau }_0)}^{\text{T}}$ be the true value of ξ = (α^T, τ)^T under H₀: β = 0. Let ${\widehat{\mathit{\psi }}}_0={({\widehat{\mathit{\xi }}}_0^{\text{T}},\mathbf{0})}^{\text{T}}$ be the restricted MPLE of ${\mathit{\psi }}_0={({\mathit{\alpha }}_0^{\text{T}},{\tau }_0,{\mathbf{0}}^{\text{T}})}^{\text{T}}={({\mathit{\xi }}_0^{\text{T}},{\mathbf{0}}^{\text{T}})}^{\text{T}}$ under H₀: β = 0. That is, ψ̂₀ is the value of ψ that maximizes the ℓ_p(ψ; λ) subject to β = ψ = 0, where = [0_K×(p+1) I_K×K]^T is a (p+1+K)×K matrix for I_K×K being an identity matrix of order K.

To test H₀: β = 0, because the model parameters are estimated by maximizing the penalized log-likelihood, by analogy with the unpenalized parametric log-likelihood procedures we construct a penalized log-likelihood ratio test (PLRT) statistic as follows:

$$Q_{\lambda }=2[{\ell }_p({\widehat{\mathit{\psi }}}_{\lambda };\lambda )-{\ell }_p({\widehat{\mathit{\psi }}}_0;\lambda )],$$ (6)

where ${\ell }_p({\widehat{\mathit{\psi }}}_0;\lambda )=\ell ({\widehat{\mathit{\psi }}}_0)-(1/2)\lambda {\widehat{\mathit{\psi }}}_0^{\text{T}}\mathcal{Q}{\widehat{\mathit{\psi }}}_0=\ell ({\widehat{\mathit{\psi }}}_0)$ because ψ̂₀ = 0. The main difference between the PLRT statistic Q_λ and the deviance statistic (Hastie and Tibshirani, 1990) is that the rough penalty is not included in the deviance.

Let I(ψ) be the Fisher’s information matrix as follows:

$$\mathit{I}(\mathit{\psi })=\left[\begin{array}{cc}{\mathit{I}}_{\mathit{\xi \xi }}(\mathit{\psi })& {\mathit{I}}_{\mathit{\xi \beta }}(\mathit{\psi })\\ {\mathit{I}}_{\mathit{\beta \xi }}(\mathit{\psi })& {\mathit{I}}_{\mathit{\beta \beta }}(\mathit{\psi })\end{array}\right]=E\left[-\frac{{\partial }^2\ell (\mathit{\psi })}{\partial \mathit{\psi }\partial {\mathit{\psi }}^{\text{T}}}\right]=\sum _{i=1}^n{b}^{″}({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi }){\mathit{A}}_{{\mathit{z}}_i}^{\text{T}}{\mathit{A}}_{{\mathit{z}}_i}.$$

Let ${\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}(\mathit{\psi })={\mathit{I}}_{\mathit{\beta \beta }}(\mathit{\psi })-{\mathit{I}}_{\mathit{\beta \xi }}(\mathit{\psi }){\mathit{I}}_{\mathit{\xi \xi }}^{-1}(\mathit{\psi }){\mathit{I}}_{\mathit{\xi \beta }}(\mathit{\psi })$. The asymptotic results of Q_λ are stated in the following theorem and the proof of the theorem is given in the Appendix.

Theorem 1

Assume that conditions (C₁) and (C₂) in the Appendix are satisfied. Let λ_n be the sequence of smoothing parameters with $\underset{n\to \infty }{lim}{\lambda }_n/n$ finite, which may be 0. When the number of knots is fixed, under H₀

$$Q_{\lambda }\stackrel{D}{\to }Q=\sum _j{\gamma }_j{\chi }_j^2,$$ (7)

where the ${\chi }_j^2$s are independent chi-squared random variables each with one degree of freedom, and the γ_js are the eigenvalues of $\underset{n\to \infty }{lim}{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0){[{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{G}]}^{-1}$

It can be seen from this theorem that for any α ∈ (0, 1) the test rejecting H₀ if the observed value of Q_λ exceeds the 100(1 − α)th percentile of the distribution of Q has an asymptotic significance level α. There have been several methods developed for calculation of the distribution of Q by, e.g., Imhof (1961), Davies (1973, 1980), Sheil and O’Muircheartaigh (1977), and Castaño-Martínex and López-Blázquez (2005). The algorithm of Davies (1980) is implemented in this work.

Because $Q_{\lambda }\stackrel{D}{\to }Q$ under H₀, one can have the asymptotically expected value of Q_λ under H₀ as follows:

$$\sum {\gamma }_j=\text{trace}\left\{\underset{n\to \infty }{lim}{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0){[{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{G}]}^{-1}\right\}.$$ (8)

The asymptotically expected value would be the degrees of freedom of the test for the case of unpenalized log-likelihood (i.e., λ = 0). Although the Q_λ in (6) does not have an asymptotic chi-squared distribution when λ ≠ 0, for simplicity and, hence, by following other researchers (e.g., Buja, Hastie, and Tibshirani, 1989), we adopt the terminology, degrees of freedom, for the test. We do not specify a value for the λ directly. Instead, in practice we give a specified value for the degrees of freedom, replace ψ₀ with ψ̂₀ in (8), and find the value of λ, denoted by λ̂, which gives the specified degrees of freedom. Therefore, Q_λ̂ = 2[ℓ_p(ψ̂_λ̂; λ̂) − ℓ_p(ψ̂₀; λ̂)] is used as a test statistic.

3. Simulation study

We conducted a Monte Carlo simulation study to assess the performance of the proposed PLRT in Section 2. We assumed the covariate vector takes the form z_i = (x_i, u_i), i = 1, …, 100. Here x_i = (x_i0, x_i1), where x_i0 = 1 and x_i1 were generated from the uniform [0.5, 1.5] distribution. u_i were generated from the uniform [0, 1] distribution. One thousand replications were conducted for each experimental configuration and all tests used nominal level α = 0.05.

To assess the empirical power of the test, we generated data y_i from the following model

$$log[\mu ({\mathit{z}}_i;\mathit{\xi },b)]={\alpha }_0+{\alpha }_1{x}_{1i}+s(u_i;\tau ,b),i=1,2,\dots ,100.$$ (9)

Here ξ = (α^T, τ)^T for α = (α₀, α₁) = (1, 0.7)^T. s(u_i; τ, b) = τu_i + b cos(3πu_i), where τ = 1.5 and b cos(3πu_i) are used as alternatives for b = 0, 0.02, …, 0.28. It is noted that when b = 0, the model in (9) produces the null model

$$log[\mu ({\mathit{z}}_i)]={\alpha }_0+{\alpha }_1{x}_{1i}+\tau u_i=log[\mu ({\mathit{z}}_i;\mathit{\xi },0)].$$ (10)

The goal was to test H₀: s(u) = τu = s(u; τ, 0), which is equivalent to testing b = 0.

We also computed two log-likelihood ratio test (LRT) statistics for H₀: b = 0 to compare their power performances with the PLRT. The first LRT statistic for H₀ is $Q_1=2[\ell (\widehat{\mathit{\xi }},\widehat{b})-\ell (\stackrel{\sim }{\mathit{\xi }})]\stackrel{D}{\to }{\chi }^2(1)$ that has a chi-squared distribution with one degree-of-freedom. Here (ξ̂, b̂) is the value of (ξ, b) maximizing the $\ell (\mathit{\xi },b)={\sum }_{i=1}^n[y_i({\mathit{z}}_i\mathit{\xi }+bcos(3\pi u_i))-exp({\mathit{z}}_i\mathit{\xi }+bcos(3\pi u_i))]$ for the data generating model in (9). ξ̃ is the value of ξ maximizing the $\ell (\mathit{\xi })={\sum }_{i=1}^n[y_i({\mathit{z}}_i\mathit{\xi })-exp({\mathit{z}}_i\mathit{\xi })]$ for the null model in (10). The test gives an asymptotically optimal test that provides a useful comparison benchmark but is of limited practical value because its use requires very specific knowledge of the form of the alternative. We rejected H₀ at α = 0.05 when the observed value of Q₁ exceeded ${\chi }_{0.95}^2(1)$ that denotes the 0.95-quantile of the chi-squared distribution with one degree-of-freedom.

To examine the power loss when the alternative parametric model is mis-specified, we fit data generated from the model in (9) by the misspecified alternative parametric model

$$log[\mu ({\mathit{z}}_i;\mathit{\xi },c)]={\alpha }_0+{\alpha }_1{x}_{1i}+\tau u_i+{cu}_i^2.$$ (11)

The second LRT statistic for H₀ is $Q_2=2[\ell (\stackrel{⌣}{\mathit{\xi }},\stackrel{⌣}{c})-\ell (\stackrel{\sim }{\mathit{\xi }})]\stackrel{D}{\to }{\chi }^2(1)$. Here (ξ̆, c̆) is the value of (ξ, c) maximizing the $\ell (\mathit{\xi },c)={\sum }_{i=1}^n[y_i({\mathit{z}}_i\mathit{\xi }+{cu}_i^2)-exp({\mathit{z}}_i\mathit{\xi }+{cu}_i^2)]$ for the misspecified model in (11). H₀ was rejected at α = 0.05 when the observed value of Q₂ exceeded ${\chi }_{0.95}^2(1)$.

To study the type I error and power performance of the PLRT Q_λ under the different numbers of knots and different degrees of freedom, we considered 4, 4.5, and 5 degrees of freedom, each with q = 10, 15, 20, and 25 knots, to determine the value of λ while fitting the B-spline model. Under each of the combinations of different degrees of freedom and the different numbers of knots, the empirical significance level of the PLRT appeared to agree with the 0.05 nominal level. The empirical powers are shown in Figure 1. The power performances of the PLRTs, among the aforementioned combinations, are essentially the same. The first LRT, Q₁, had the best power performance; however, the PLRT was competitive with the test and still retained the advantage of having the ability to detect more general alternatives. The second LR test, Q₂, suffered a severe power loss and had the worst power performance among these tests; thus, it indicated the parametric test Q₂ was not optimal under the case where a misspecified parametric regression model was used to fit data.

Figure 1.

Empirical power curve comparison of the PLRT (d; q), LRT₁ (i.e., Q₁), and LRT₂ (i.e., Q₂) for the null model s(u) = τu versus the alternative model s(u) = τu + b cos(3πu). Abbreviations: PLRT (d; q), penalized log-likelihood ratio test associated d degrees of freedom and q knots; LRT₁, log-likelihood ratio test with data generating model s(u) = τu + b cos(3πu) as alternative model; LRT₂, log-likelihood ratio test with misspecified model s(u) = τu + cu² as alternative model.

4. Example

We illustrate the practical use of the methodology with the data set from a study of 173 nesting horseshoe crabs (Brockmann, 1996; Agresti, 2002). In the study, each female horseshoe crab had a male crab attached to her in her nest. The study investigated factors that affect whether the female crab had any other males, called satellites, residing nearby her. Explanatory variables included the female crab’s color, which is light medium, medium, dark medium, or dark, spine condition, which is both good, one worn or broken, or both worn or broken, carapace width (W) in centimeters, and weight (Wt) in kgs. The response outcome for each female crab is her number of satellites (Sa).

We fit the Poisson log-linear regression to the data to produce the fitted log-linear regression model log μ̂ = −0.801 + 0.531 × I_{{light medium}} + 0.266 × I_{medium} + 0.018 × I_{{dark medium}} − 0.087 × I_{{both good}} − 0.238 × I_{{one worn or broken}} + 0.017 × W + 0.497 × Wt, where $\widehat{\mu }=\widehat{E(Sa)}$. We employ the PLRT to evaluate whether there is a linear relation between the logarithm of the expected number of satellites, log μ = log E(Sa), and female crab’s weight. Because the p-values of the PLRT associated with 4, 4.5, and 5 degrees of freedom, each calculated by using 10, 15, 20, and 25 knots, are less than 0.0035, the female crab’s weight has a statistically significantly nonlinear effect on the logarithm of the expected number of satellites. Figures 2 and 3 show a cubic spline-fitted curve for the effect of female crab’s weight obtained with each of the above combinations of different degrees of freedom and the different numbers of knots.

Figure 2.

Cubic spline-fitted curves for the effect of female crab’s weight by using 4, 4.5, and 5 degrees of freedom with 10, 15, 20, and 25 knots.

Figure 3.

Cubic spline-fitted curves for the effect of female crab’s weight by using 4, 4.5, and 5 degrees of freedom with 10, 15, 20, and 25 knots.

5. Discussion

A flexible method has been proposed for modeling the nonparametric component of the semiparametric generalized linear model in (2) via a linear combination of fixed-knot cubic B-splines with the second-order difference penalty on the adjacent cubic B-spline coefficients to prevent overfitting to the data. Although automatic smoothing parameter selection methods such as cross-validation and generalized cross-validation, can be considered in the estimation procedure, we give a specified value for the degrees of freedom, and find the value of λ that gives the specified degrees of freedom. For comments on using fixed smoothing parameters, see, e.g., Buja et al. (1989, p. 545–546).

The focus is only on testing H₀: s(u) = τu, but the idea also can be used to test the null hypothesis of no effect of u, H₀: s(u) = 0, i.e., (τ, β^T)^T = 0. It is exactly the same as constructing the PLRT statistic for H₀: s(u) = τu to construct one for H₀: s(u) = 0. One can extend the methodology to examine the effects of several covariates of interest on the response variable.

Appendix

In this appendix, we set out the essential steps for proving Theorem 1. The following conditions are required.

• (C₁) $\underset{n\to \infty }{lim}\frac{1}{n}\mathit{I}(\mathit{\psi })$ is finite and positive definite.

• (C₂) $\underset{{\mathit{\psi }}^{\ast }\in B(\delta )}{sup}\frac{1}{n}{\displaystyle \sum _{i=1}^n}\Vert [b^{″}({\mathit{A}}_{{\mathit{z}}_i}{\mathit{\psi }}^{\ast })-b^{″}({\mathit{A}}_{{\mathit{z}}_i}\mathit{\psi })]{\mathit{A}}_{{\mathit{z}}_i}^{\text{T}}{\mathit{A}}_{{\mathit{z}}_i}\Vert \to 0$ as δ ↓ 0, where B(δ) = {ψ^* ∈ R^p+1+K: ||ψ^* − ψ|| < δ}, δ ↓ 0 is an infinitesimal neighborhood of ψ.

Let ${\widehat{\mathit{\psi }}}_{{\lambda }_n}={({\widehat{\mathit{\xi }}}_{{\lambda }_n}^{\text{T}},{\widehat{\mathit{\beta }}}_{{\lambda }_n}^{\text{T}})}^{\text{T}}$ be the value of ψ that maximizes ℓ_p(ψ; λ) = ℓ(ψ) − (1/2)λψ^T ψ for a sample of size n when λ = λ_n. Since ∂ℓ_p(ψ; λ_n)/∂ψ|_ψ=ψ̂λn = 0, by using (C₁), we consider a Taylor expansion of ∂ℓ_p(ψ; λ_n)/∂ψ|_ψ=ψ̂λn around ψ₀ to have $\mathbf{0}=\partial {\ell }_p(\mathit{\psi };{\lambda }_n)/\partial \mathit{\psi }{\mid }_{\mathit{\psi }={\mathit{\psi }}_0}+{\partial }^2{\ell }_p(\mathit{\psi };{\lambda }_n)/(\partial \mathit{\psi }\partial {\mathit{\psi }}^{\text{T}}){\mid }_{\mathit{\psi }={\mathit{\psi }}_0}({\widehat{\mathit{\psi }}}_{{\lambda }_n}-{\mathit{\psi }}_0+{\mathit{o}}_p(\sqrt{n})$, where o_p(a_n) indicates a vector whose components are uniformly o_p(a_n). Because ψ = 0 under H₀: β = 0, ∂ℓ_p(ψ; λ_n)/∂ψ|_ψ=ψ0 = ∂ℓ(ψ)/∂ψ|_ψ=ψ0 = U (ψ₀). Thus, it implies that under H₀

$$\sqrt{n}({\widehat{\mathit{\psi }}}_{{\lambda }_n}-{\mathit{\psi }}_0)=n{[\mathit{I}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{Q}]}^{-1}{n}^{-1/2}\mathit{U}({\mathit{\psi }}_0)+{\mathit{o}}_p(1).$$ (12)

By using the assumption that lim_n→∞λ_n/n is finite, Liapounov’s Central Limit Theorem, and Cramér-Wold Theorem, one can show $n^{-1/2}\mathit{U}({\mathit{\psi }}_0)\stackrel{D}{\to }{N}_{p+1+K}(\mathbf{0},{lim}_{n\to \infty }{n}^{-1}\mathit{I}({\mathit{\psi }}_0))$ under H₀. Therefore, using Slutsky’s Theorem and (C₂) can show $\sqrt{n}({\widehat{\mathit{\psi }}}_{{\lambda }_n}-{\mathit{\psi }}_0)\stackrel{D}{\to }{N}_{p+1+K}(\mathbf{0},{lim}_{n\to \infty }{n\mathit{V}}_{{\lambda }_n}({\mathit{\psi }}_0))$ under H₀, where V_λn(ψ₀) = [I(ψ₀) + λ_n ]⁻¹I(ψ₀)[I(ψ₀) + λ_n ]⁻¹. Then,

$$\sqrt{n}{\widehat{\mathit{\beta }}}_{{\lambda }_n}={\mathcal{L}}^{\text{T}}\sqrt{n}({\widehat{\mathit{\psi }}}_{{\lambda }_n}-{\mathit{\psi }}_0)\stackrel{D}{\to }{N}_K\left(\mathbf{0},\underset{n\to \infty }{lim}{n\mathit{V}}_{\mathit{\beta \beta };{\lambda }_n}({\mathit{\psi }}_0)\right),$$ (13)

where V_ββ;λn(ψ₀) = [I_ββ|ξ (ψ₀) + λ_n ]⁻¹I_ββ|ξ(ψ₀)[I_ββ|ξ(ψ₀) + λ_n ]⁻¹.

Under H₀: ψ = β = 0, ψ̂₀ is the restricted MPLE of ψ₀, so ψ̂₀ must satisfy the restricted penalized log-likelihood equations

$$\begin{array}{c}{\frac{\partial }{\partial \mathit{\psi }}\left\{{\ell }_p(\mathit{\psi };{\lambda }_n)+({\mathcal{L}}^{\text{T}}\mathit{\psi }-\mathbf{0})\mathit{d}\right\}|}_{\mathit{\psi }={\widehat{\mathit{\psi }}}_0}={\frac{\partial }{\partial \mathit{\psi }}{\ell }_p(\mathit{\psi };{\lambda }_n)|}_{\mathit{\psi }={\widehat{\mathit{\psi }}}_0}+\mathcal{L}\mathit{d}=\mathbf{0},\\ {\mathcal{L}}^{\text{T}}{\widehat{\mathit{\psi }}}_0=\mathbf{0},\end{array}$$ (14)

where d is a K ×1 vector of Lagrange multipliers. We use (C₁) and ψ₀ = 0 to consider the Taylor expansion $n^{1/2}{\ell }_p(\mathit{\psi };{\lambda }_n)/\partial \mathit{\psi }{\mid }_{\mathit{\psi }={\widehat{\mathit{\psi }}}_0}=n^{1/2}\mathit{U}({\mathit{\psi }}_0)-n^{-1}[\mathit{I}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{Q}]\sqrt{n}({\widehat{\mathit{\psi }}}_0-{\mathit{\psi }}_0)+{\mathit{o}}_p(1)$. In addition, $\sqrt{n}{\mathcal{L}}^{\text{T}}{\widehat{\mathit{\psi }}}_0=\sqrt{n}{\mathcal{L}}^{\text{T}}({\widehat{\mathit{\psi }}}_0-{\mathit{\psi }}_0)$. One can then express $\sqrt{n}({\widehat{\mathit{\psi }}}_0-{\mathit{\psi }}_0)$ as

$$\sqrt{n}({\widehat{\mathit{\psi }}}_0-{\mathit{\psi }}_0)=n\widehat{\mathcal{L}}{[{\widehat{\mathcal{L}}}^{\text{T}}(\mathit{I}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{Q})\widehat{\mathcal{L}}]}^{-1}{\widehat{\mathcal{L}}}^{\text{T}}{n}^{-1/2}\mathit{U}({\mathit{\psi }}_0)+{\mathit{o}}_p(1),$$

where = [I_(p+1)×(p+1) 0_K×(p+1)]^T is a (p+1+K)×(p+1) matrix. Because by using a second-order Taylor expansion of ℓ_p(ψ̂_λn; λ_n) and ℓ_p(ψ̂₀; λ_n), one can have 2[ℓ_p(ψ̂_λn; λ_n) − ℓ_p(ψ₀; λ_n)] = U ^T(ψ₀)[I(ψ₀)+ λ_n ]⁻¹U(ψ₀)+ o_p(1) and 2[ℓ_p(ψ̂₀; λ_n)−ℓ_p(ψ₀; λ_n)] = U^T(ψ₀) { [I(ψ₀)+λ_n ] }⁻¹ U(ψ₀)+ o_p(1) under H₀, using (12) can express $Q_{{\lambda }_n}=2[{\ell }_p({\widehat{\mathit{\psi }}}_{{\lambda }_n};{\lambda }_n)-{\ell }_p({\widehat{\mathit{\psi }}}_0;{\lambda }_n)]={\widehat{\mathit{\beta }}}_{{\lambda }_n}^{\text{T}}[{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{G}]{\widehat{\mathit{\beta }}}_{{\lambda }_n}+o_p(1)$. By means of a non-singular linear transformation (Scheffé, 1959, p. 418) and Slutsky’s Theorem, one can show $Q_{{\lambda }_n}\stackrel{D}{\to }Q$ under H₀. Let λ̂ be an estimator satisfying $\widehat{\lambda }/{\lambda }_n\stackrel{p}{\to }1$. From (13), we can show via Slutsky’s Theorem that $\sqrt{n}{\widehat{\mathit{\beta }}}_{\widehat{\lambda }}\stackrel{D}{\to }{N}_K(\mathbf{0},{lim}_{n\to \infty }{n\mathit{V}}_{\mathit{\beta \beta };{\lambda }_n}({\mathit{\psi }}_0))$ under H₀. Then, β̂_λ̂ and β̂_λn have the same limiting null distribution. Let $Q_{1{\lambda }_n}={\widehat{\mathit{\beta }}}_{{\lambda }_n}^{\text{T}}[{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0)+{\lambda }_n\mathcal{G}]{\widehat{\mathit{\beta }}}_{{\lambda }_n}$ and $Q_{1\widehat{\lambda }}={\widehat{\mathit{\beta }}}_{\widehat{\lambda }}^{\text{T}}[{\mathit{I}}_{\mathit{\beta \beta }\mid \mathit{\xi }}({\mathit{\psi }}_0)+\widehat{\lambda }\mathcal{G}]{\widehat{\mathit{\beta }}}_{\widehat{\lambda }}$. It follows via the Slutsky’s Theorem that Q_λn = Q_1λn + o_p(1), Q_1λn and Q_1λ̂ have the same limiting null distribution. Thus, one can show that Q_λ̂ = 2[ℓ_p(ψ̂_λ̂; λ̂) − ℓ_p(ψ̂₀; λ̂)] = Q_1λ̂ + o_p(1), Q_1λ̂, Q_1λn, and Q_λn have the same limiting null distribution.