Sieve Maximum Likelihood Estimation for Doubly Semiparametric Zero-Inflated Poisson Models

Abstract

For nonnegative measurements such as income or sick days, zero counts often have special status. Furthermore, the incidence of zero counts is often greater than expected for the Poisson model. This article considers a doubly semiparametric zero-inflated Poisson model to fit data of this type, which assumes two partially linear link functions in both the mean of the Poisson component and the probability of zero. We study a sieve maximum likelihood estimator for both the regression parameters and the nonparametric functions. We show, under routine conditions, that the estimators are strongly consistent. Moreover, the parameter estimators are asymptotically normal and first-order efficient, while the nonparametric components achieve the optimal convergence rates. Simulation studies suggest that the extra flexibility inherent from the doubly semiparametric model is gained with little loss in statistical efficiency. We also illustrate our approach with a dataset from a public health study.

1 Introduction

Poisson regression models are frequently used in the analysis of count data. In practice, however, the incidence of zero counts is sometimes greater than expected for the Poisson distribution, and zero has special connotations. For example, in a public health study of Lam, Xue and Cheung (2006), the number of days that people of working age miss their primary activities due to illness in a four-week period exhibits a substantially large proportion of zeros. Failure to account for over-dispersion of this type can lead to serious underestimation of variance for the regression parameters.

In recent years, several models and associated estimation methods have been proposed for handling such data in the biomedical and econometric literature. The zero-inflated Poisson (ZIP) regression model proposed by Mullahy (1986) and developed by Lambert (1992) is among the widely used, but earlier work focused on the parametric ZIP model. Lam, Xue and Cheung (2006) extends it to a semiparametric ZIP model by replacing the linear regression function with a partly linear regression function for the mean of the Poisson distribution, while maintaining a linear regression function in modeling the probability of zero. It is not difficult to consider partly linear regression functions both for the mean of Poisson distribution and in the probability of zero. With two nonparametric regression functions involved, we have a doubly semiparametric ZIP model. In fact, Lam, Xue and Cheung (2006) suggested this possibility in their concluding section, but did not develop in that direction. Chiogna and Gaetan(2007) proposed a general additive model and penalized maximum likelihood estimation based on regression splines, but the large-sample properties of their estimators are not yet developed. In this article we consider sieve maximum likelihood estimation (MLE) for the doubly semiparametric ZIP model and establish its asymptotic properties. When two or more nonparametric functions are involved, we are unable to use the standard information calculation based on orthogonal projections, so a different mathematical approach from that of Lam, Xue and Cheung (2006) is taken in the present paper. We show that the extra flexibility from allowing nonparametric functions for some of the covariates is gained at little cost in efficiency for estimating the regression parameters.

Alternative models to ZIP have also been considered in the literature, including the Poisson hurdle model used in Mullahy (1986), Miaou (1994), Bohara and Krieg (1996). A comparison of various models can be found in Welsh et al. (1996). Both the Poisson hurdle model and the ZIP model treat the response as having two states. At the first state, a binary random effect is assumed to categorize the dichotomy of the population into two subpopulations: the low-risk group and the high-risk group. But at the second state, a zero truncated Poisson distribution is used in the Poisson hurdle model, whereas the classical Poisson distribution is used in the ZIP model. The zero counts arise only from the low-risk group for the Poisson hurdle models, but they can arise from both the low-risk group and the high-risk group for the ZIP models. One attraction of the ZIP model is that it is able to pick up two different regimes from which the zero counts arise, which is often desirable in biometrical research. We believe that both models are useful, but this article focuses on the ZIP model whose analysis is more challenging mathematically, because two regimes in the ZIP model are not clearly separable from the sample.

The rest of the article is organized as follows. In Section 2 we describe the doubly semiparametric ZIP model and a sieve maximum likelihood estimator (MLE). We outline asymptotic properties of the sieve MLE in Section 3, and derive the information bound in Section 4. Simulation studies are given in Section 5 to illustrate the advantages of the doubly semiparametric ZIP model when simpler models are inadequate. We present an illustrative example in Section 6 for the proposed approach, with some concluding remarks in Section 7.

Throughout this article we let ‖a‖ be the Euclidean norm (or L₂ norm) of a vector a, I_(A) be the indicator function of the set A and ${\Vert f\Vert }_{\infty }=\underset{t}{\text{sup}}|f(t)|$ be the supremum norm of a function f. Moreover for a random vector X ~ P where P is a probability measure, we let ${\Vert f(X)\Vert }_2={\Vert f\Vert }_{P,2}={({\displaystyle \int f^2\mathit{\text{dP}}})}^{\frac{1}{2}}$ be the L₂(P)-norm of a function f, and we adopt the short notation Pf for the expected value of f(X).

2 Models and Estimation

Suppose that the counts, Y, are generated independently according to a zero-inflated Poisson distribution; the zeros are assumed to arise from two distinct states. The first state occurs with probability p and produces structural zeros, while the other state occurs with probability (1 − p) and leads to a standard Poisson count with mean λ (Jansakul and Hinde, 2002). The zeros from the Poisson distribution are called sampling zeros, which occur by chance. This two-state process gives a simple two-component mixture distribution with probability mass function

$$\mathit{\text{Pr}}(Y=y)=\mathbf{\{}\begin{array}{ccc}p+(1-p)e^{-\lambda },\hfill & y=0,\hfill & \hfill \\ (1-p)\frac{e^{-\lambda }{\lambda }^y}{y!},\hfill & y=1,2,\cdots ,\hfill & 0\le p\le 1,\hfill \end{array}$$ (1)

and

$$E(Y)=(1-p)\lambda =\mu \text{ and }\mathit{\text{var}}(Y)=\mu +(\frac{p}{1-p}){\mu }^2.$$

To use the zero-inflated Poisson model with covariates, Lambert (1992) suggested the following joint models for λ and p

$$\text{log}(\lambda )={\mathit{\beta }}^{*T}{(1,{\mathit{X}}^T)}^T\text{ and }\log (\frac{p}{1-p})={\mathit{\gamma }}^{*T}{(1,{\mathit{Z}}^T)}^T,$$ (2)

where X = (X₁, ⋯,X_d1)^T and Z = (Z₁, ⋯,Z_d2)^T are the vectors of covariates, and β = (β₀, β₁, ⋯, β_d1)^T, γ = (γ₀, ⋯, γ_d2 )^T are (d₁ + 1)- and (d₂ + 1)-dimensional vectors of unknown regression parameters, respectively.

Lam, Xue and Cheung (2006) extend the above parametric ZIP model to a semiparametric one with a partially linear link function for λ:

$$\text{log}(\lambda )={\mathit{\beta }}^T\mathit{X}+g(T)\text{ and }\text{log}(\frac{p}{1-p})={\mathit{\gamma }}^T{(1,{\mathit{X}}^T,T)}^T,$$ (3)

where T is another covariate that enters the mean function of the Poisson distribution with a nonparametric function g. In our notation, β = (β₁, ⋯, β_d1)^T and γ = (γ₀, ⋯, γ_d1+1)^T will be d₁- and (d₁ +2)-dimensional regression parameters.

It is somewhat unnatural that T contributes non-parametrically to λ but linearly to p. In this article, we extend it to the doubly semiparametric ZIP model

$$\text{log}(\lambda )={\mathit{\beta }}^T\mathit{X}+g(T)\text{ and }\text{log}(\frac{p}{1-p})={\mathit{\gamma }}^T\mathit{X}+h(T),$$ (4)

where both g and h are unknown smooth functions, but β = (β₁, ⋯, β_d1)^T and γ = (γ₁, ⋯, γ_d1)^T are d₁-dimensional regression parameters.

To fix notation, let W = (Y,X^T,T)^T be the data vector, and θ = (β^T, γ^T, g, h)^T be the vector of all the unknown quantities of interest with ${\mathit{\theta }}_0={({\mathit{\beta }}_0^T,{\mathit{\gamma }}_0^T,g_0,h_0)}^T$ as the true value of θ. Also let

$$\begin{array}{c}{B}_g=\{g\in C^r[0,1]:-\infty <m_0\le g(t)\le M_0<+\infty ,\forall t\in [0,1]\},\hfill \\ B_h=\{h\in C^r[0,1]:-\infty <m_1\le h(t)\le M_1<+\infty ,\forall t\in [0,1]\},\hfill \end{array}$$

where r = 1 or 2, and (m₀,M₀,m₁,M₁) are any fixed constants. Since the asymptotic theory in this article works equally for r = 1 and r = 2, we suppress the dependence of B_g and B_h in the notation here. To further facilitate our asymptotic study, we assume that we have the parameter space Θ = {θ : β ∈ A₁, γ ∈ A₂, g ∈ B_g, h ∈ B_h} = A₁ * A₂ * B_g * B_h, where A₁ and A₂ are compact sets in R^d₁. The density function of W is thus given by

$$Q(\mathit{w},\mathit{\theta })=\left\{I_{(Y=0)}[p+(1-p)e^{-\lambda }]+I_{(Y>0)}(1-p)\frac{e^{-\lambda }{\lambda }^y}{y!}\right\}\phi (x,t)$$

where φ is the joint density function of (X^T, T), and λ and p are defined in (4). Because φ(X, T) is independent of the model parameters, it can be set aside in the estimation of θ. Then for any observation w of W, the log-likelihood function can be represented as

$$\begin{array}{cc}\ell \hfill & =\ell (\mathit{\theta },\mathit{w})=\text{log}Q(\mathit{w},\mathit{\theta })\hfill \\ \hfill & =I_{(y=0)}\text{log}[p+(1-p)e^{-\lambda }]+I_{(y>0)}[\text{log}(1-p)-\lambda +y\text{log}(\lambda )].\hfill \end{array}$$

Suppose that we have a random sample W̃ = (W₁,⋯,W_n)^T, with P_θ denoting the distribution of W under the parameter θ. For convenience, we will use E₀ for expectation with respect to P_θ0, and P_n for the empirical distribution of W̃. Furthermore, for any θ_i ∈ Θ, i = 1, 2, define a distance by

$$d({\mathit{\theta }}_1,{\mathit{\theta }}_2)=\Vert {\mathit{\beta }}_1-{\mathit{\beta }}_2\Vert +\Vert {\mathit{\gamma }}_1-{\mathit{\gamma }}_2\Vert +{\Vert g_1-g_2\Vert }_2+{\Vert h_1-h_2\Vert }_2.$$ (5)

To make inference about θ, we propose the sieve method to approximate an infinite dimensional parameter space Θ by a series of finite dimensional parameter spaces Θ_n. Let 0 = t₀ < t₁ < ⋯ < t_m = 1 define a partition of [0, 1]. Choose m = m_n to be an integer that grows at rate n^k for 0 < k < 1. A more precise range for k required for the asymptotic results will be given in Section 3. In practice, the number of knots, m + 1, can be chosen by the Akaike information criterion (AIC) as illustrated in Sections 5 and 6. Let I_j(t) = I_{(tj−1≤t<tj)} for 1 ≤ j ≤ m − 1, and I_m(t) = I_{(tm−1≤t≤tm)}. We approximate g(t) and h(t) by piecewise linear functions in the form of

$$G_m(t;\mathit{u})={\displaystyle \sum _{j=1}^m\left(\frac{u_j-u_{j-1}}{t_j-t_{j-1}}t-\frac{u_j{t}_{j-1}-u_{j-1}{t}_j}{t_j-t_{j-1}}\right)I_j(t),}$$ (6)

where u = (u₀,⋯,u_m)^T is the parameter vector for G_m(.;u). In particular, let b = (g(t₀), ⋯,g(t_m))^T, a = (h(t₀), ⋯, h(t_m))^T, g_n(.) = G_m(.; b), and h_n(.) = G_m(.; a). For any θ = (β^T, γ^T, g, h)^T ∈ Θ, we define the mapping π_nθ = (β^T, γ^T, g_n, h_n)^T ∈ Θ_n where Θ_n = A₁ * A₂ * B_g,n * B_h,n is a product space, and

$$\begin{array}{c}{B}_{g,n}=\{G_m(t;\mathit{b}):m_0\le b_i\le M_0,0\le i\le m\},\hfill \\ B_{h,n}=\{G_m(t;\mathit{a}):m_1\le a_i\le M_1,0\le i\le m\}.\hfill \end{array}$$

With the piecewise linear approximations, we have

$$d({\mathit{\pi }}_{\mathit{n}}\mathit{\theta },\mathit{\theta })\le {\Vert g-g_n\Vert }_{\infty }+{\Vert h-h_n\Vert }_{\infty }=O(n^{-\mathit{\text{rk}}})\to 0\text{ as }n\to \infty .$$ (7)

This motivates us to select Θ_n = A₁ * A₂ * B_g,n * B_h,n as a sieve space of Θ. Furthermore, if we let $L_n(\mathit{\theta },\tilde{\mathit{W}})=P_n\ell (\mathit{\theta },\mathit{w})=\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\ell (\mathit{\theta },{\mathit{w}}_{\mathit{i}})}$ as the empirical objective function, then

$${\widehat{\mathit{\theta }}}_{\mathit{n}}={({\widehat{\mathit{\beta }}}_{\mathit{n}}^{\mathit{T}},{\widehat{\mathit{\gamma }}}_{\mathit{n}}^{\mathit{T}},{\widehat{g}}_n,{\widehat{h}}_n)}^T=\text{arg}\underset{\mathit{\theta }\in {\mathrm{\Theta }}_n}{\text{sup}}{L}_n(\mathit{\theta },\tilde{\mathit{W}})$$

is the sieve MLE for θ₀; see Grenander (1981) and Shen and Wong (1994) for more details on sieve MLEs in general. This sieve MLE will be studied in the present paper.

3 Asymptotic Properties

In the Appendix, we give Assumptions C1 – C3, and A1 – A4, for the asymptotic results discussed in this paper. Under Assumptions C1 and C2, the proposed model is identifiable by similar arguments in Li, Taylor and Sy(2001). Two consistency results are summarized first.

Theorem 1 Under Assumptions C1–C3, we have (1) Strong Consistency d(θ̂_n, θ₀) → 0, almost surely under P_θ0; and (2) (Rate of Convergence) d(θ̂_n, θ₀) = O_p(n^−(1−k)/2+ n^−rk).

If we select k = 1/(1 + 2r) for r = 1 or 2, d(θ̂_n, θ₀) achieves the optimal nonparametric convergence rate O_p(n^−r/(1+2r)) under the smoothness conditions imposed on h and g.

To calculate the asymptotic variance of the sieve MLE, we need to introduce additional notation for the sake of presentation. Specifically, let

$$\begin{array}{c}\hfill \xi (\mathit{\theta },\mathit{W})={\mathit{\beta }}^T\mathit{X}+g(T),\eta (\mathit{\theta },\mathit{W})={\mathit{\gamma }}^T\mathit{X}+h(T),\hfill \\ \hfill D_1(\xi )=\frac{\partial }{\partial \xi }\ell (\mathit{\theta },\mathit{W})=-I_{(y=0)}\frac{(1-p)e^{\xi -e^{\xi }}}{p+(1-p)e^{-e^{\xi }}}+I_{(y>0)}(y-e^{\xi }),\hfill \\ \hfill D_2(\eta )=\frac{\partial }{\partial \eta }\ell (\mathit{\theta },\mathit{W})=\frac{\partial \ell }{\partial p}\frac{\partial p}{\partial \eta }=\{I_{(y=0)}\frac{1-e^{-e^{\xi }}}{p+(1-p)e^{-e^{\xi }}}-I_{(y>0)}\frac{1}{1-p}\}\frac{e^{\eta }}{{(1+e^{\eta })}^2},\hfill \end{array}$$

and

$$J(\mathit{W})=C_1^{-1}{D}_2^2(\eta ({\mathit{\theta }}_{\mathbf{0}},\mathit{W}))Q(\mathit{W},{\mathit{\theta }}_{\mathbf{0}}),\mathit{\text{ with }}{C}_1=E_0{D}_2^2(\eta ({\mathit{\theta }}_{\mathbf{0}},\mathit{W})),$$

Note that J is a probability density function, so we shall use E_J for the expectation with respect to J. In addition, let

$$\begin{array}{c}\hfill \delta (\mathit{W},\mathit{\theta })=\frac{D_1(\xi )}{D_2(\eta )}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}|T\},\hfill \\ \hfill H(\mathit{W},{\mathit{\theta }}_{\mathbf{0}})=C_2^{-1}{E}_0{D}_2^2(\eta ({\mathit{\theta }}_{\mathbf{0}},\mathit{W})){\delta }^2(\mathit{W},{\mathit{\theta }}_{\mathbf{0}})Q(\mathit{W},{\mathit{\theta }}_{\mathbf{0}}),\hfill \end{array}$$

where $C_2=E_0{D}_2^2(\eta ({\mathit{\theta }}_{\mathbf{0}},\mathit{W})){\delta }^2(\mathit{W},{\mathit{\theta }}_{\mathbf{0}})$, and H is also a probability density function. As usual, we shall use E_H to be the expectation with respect to H. Therefore, the score functions for the parameters β and γ are the derivatives of the log-likelihood function given by

$${\dot{\ell }}_{\mathit{\beta }}(\mathit{\theta },\mathit{W})=\frac{\partial \ell }{\partial \mathit{\beta }}=\frac{\partial \ell }{\partial \xi }\frac{\partial \xi }{\partial \mathit{\beta }}=D_1(\xi )\mathit{X},$$ (7)

$${\dot{\ell }}_{\mathit{\gamma }}(\mathit{\theta },\mathit{W})=\frac{\partial \ell }{\partial \mathit{\gamma }}=\frac{\partial \ell }{\partial \eta }\frac{\partial \eta }{\partial \mathit{\gamma }}=D_2(\eta )\mathit{X}.$$ (8)

Theorem 2 (Efficient Score Function and Fisher Information Matrix) Under Assumption A1, in addition to C1 – C3, the efficient score function of (β, γ) is

$${\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}(\mathit{\theta },\mathit{W})=D_2(\eta )\mathbf{(}\begin{array}{c}\hfill \frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-\delta E_H\{\{\frac{D_1(\xi )}{D_2(\eta )}{\delta }^{-1}\mathit{X}\}|T\}-(1-\delta E_H({\delta }^{-1}|T))E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}\hfill \\ \hfill \mathit{X}-\delta E_H({\delta }^{-1}\mathit{X}|T)-(1-\delta E_H({\delta }^{-1}|T))E_J(\mathit{X}|T)\hfill \end{array}\mathbf{)},$$

and the Fisher information matrix is I(θ₀) = E₀(ℓ̃_β,γ,ℓ̃_β,γ^T) > 0.

The derivation of Theorem 2 is given in the next section.

Remark 1. Huang (1999) and Ma (2009) obtained efficient scores for partly linear additive Cox models and partly linear Cox cure models, but their results are not in closed form as in Theorem 2 here.

Theorem 3 (Asymptotic Normality and Efficiency) Under Assumptions A1 – A4 and C1 – C3, we have

$$\begin{array}{cc}\sqrt{n}{({\widehat{\mathit{\beta }}}_{\mathit{n}}-{\mathit{\beta }}_{\mathbf{0}},{\widehat{\mathit{\gamma }}}_{\mathit{n}}-{\mathit{\gamma }}_{\mathbf{0}})}^T\hfill & ={\mathit{I}}^{-1}({\mathit{\theta }}_{\mathbf{0}})\sqrt{n}{P}_n{\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}({\mathit{\theta }}_{\mathbf{0}},\mathit{W})+o_p(1)\hfill \\ \hfill & \stackrel{d}{\to }N(0,{\mathit{I}}^{-1}({\mathit{\theta }}_{\mathbf{0}})),\hfill \end{array}$$

and thus (β̂_n, γ̂_n)^T is asymptotically efficient.

The efficient score calculations are given in the next section, but the proofs for Theorems 1 and 3 are sketched in the appendix.

4 Information Bound Calculation

4.1. Efficient Score

In this section, we derive the information bound for the estimation of (β, γ). Due to the presence of the two nonparametric functions g and h, we are unable to use the standard information calculation based on orthogonal projections (Bickel et al., 1993). In this section, we apply a non-orthogonal projection, partially based on Sasieni (1992), which carried out the information calculation in the partly linear Cox model by a projection onto a sum-space of two nonorthogonal L₂ spaces. Huang (1999) extended this method to the partly additive Cox model with right censored data, where the partial likelihood method on polynomial splines was adopted. Ma (2009) applied it for partly linear Cox cure models with current status data, where the penalized maximum likelihood method was used. However, those studies concern a single nonparametric function or a set of additive functions in the same model, but we have two nonparametric functions that appear in two parts of the model.

Let B_g,0 = {g_τ (.) : τ in a neighborhood of 0 ∈ R} be a set of smooth curves in B_g with g_τ=0(t) = g₀(t), and let A_g be the collection of all functions $a_g=\frac{\partial }{\partial \tau }{g}_{\tau }(t){|}_{\tau =0}$, where g_τ ∈ B_g,0. Then the score operator for g is given by

$${\dot{\ell }}_g(a_g)=\frac{\partial }{\partial \tau }\ell ({\mathit{\beta }}_{\mathbf{0}},{\mathit{\gamma }}_{\mathbf{0}},g_{\tau },h_0,\mathit{W}){|}_{\tau =0}=D_1(\xi ({\mathit{\theta }}_{\mathbf{0}},\mathit{W}))a_g(T).$$ (10)

If we define B_h,0, a_h and A_h in the same way, the score operator for h is given by

$${\dot{\ell }}_h(a_h)=\frac{\partial }{\partial \tau }\ell ({\mathit{\beta }}_{\mathbf{0}},{\mathit{\gamma }}_{\mathbf{0}},g_0,h_{\tau },\mathit{W}){|}_{\tau =0}=D_2(\eta ({\mathit{\theta }}_{\mathbf{0}},\mathit{W}))a_h(T).$$ (11)

We now use 3 steps to complete the derivation.

Step 1. We first project ℓ̇_β,γ = (ℓ̇_β,ℓ̇_γ)^T onto the space generated by ℓ̇_h. Let Ṗ_h,2 be the linear span of ℓ̂ _h (a_h) in L₂(P), and ${\mathit{U}}^{\mathbf{*}}={({\mathit{U}}_{\mathit{\beta }}^{\mathbf{*}},{\mathit{U}}_{\mathit{\gamma }}^{\mathbf{*}})}^T$ be the projection of the score function ℓ̇_β,γ onto the orthocomplement of Ṗ_h,2. If there exists ${\mathit{a}}_{\mathit{h}}^{\mathbf{*}}={(a_{h,\mathit{\beta }}^{*},a_{h,\mathit{\gamma }}^{*})}^T\in {\dot{\mathbf{P}}}_{h,2}$ such that ${\dot{\ell }}_h({\mathit{a}}_{\mathit{h}}^{\mathbf{*}})$ is the projection of ℓ̇_β,γ onto the orthocomplement of Ṗ_h,2, then ${\mathit{U}}^{\mathbf{*}}={\dot{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}-{\dot{\ell }}_h({\mathit{a}}_{\mathit{h}}^{\mathbf{*}}),\text{ and }{E}_0{({\dot{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}-{\dot{\ell }}_h(a_h))}^2$ is minimized at $a_h={\mathit{a}}_{\mathit{h}}^{\mathbf{*}}$. Let

$$\begin{array}{cc}\mathrm{\Delta }(a_h)\hfill & =E_0{\{{\dot{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}-{\dot{\ell }}_h(a_h)\}}^2\hfill \\ \hfill & =E_0{D}_2^2(\eta )\mathbf{\left(}\begin{array}{c}\hfill {\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-a_{h,\mathit{\beta }}(T)\}}^2\hfill \\ \hfill {\{\mathit{X}-a_{h,\mathit{\gamma }}(T)\}}^2\hfill \end{array}\mathbf{\right)}\hfill \\ \hfill & =C_1{E}_J\mathbf{\left(}\begin{array}{c}\hfill {\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-a_{h,\mathit{\beta }}(T)\}}^2\hfill \\ \hfill {\{\mathit{X}-a_{h,\mathit{\gamma }}(T)\}}^2\hfill \end{array}\mathbf{\right)}.\hfill \end{array}$$

It follows that the least favorable direction, i.e., the minimizer of Δ(a_h), is given by

$$\begin{array}{cc}{a}_{h,\mathit{\beta }}^{*}(T)\hfill & =E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\},\hfill \\ a_{h,\mathit{\gamma }}^{*}(T)\hfill & =E_J(\mathit{X}|T).\hfill \end{array}$$

Therefore ${\mathit{a}}_{\mathit{h}}^{\mathbf{*}}(T)\in L_2(P)$, and

$${\mathit{U}}^{\mathbf{*}}=\mathbf{\left(}\begin{array}{c}\hfill {\mathit{U}}_{\mathit{\beta }}^{\mathbf{*}}\hfill \\ \hfill {\mathit{U}}_{\mathit{\gamma }}^{\mathbf{*}}\hfill \end{array}\mathbf{\right)}=D_2(\eta )\mathbf{\left(}\begin{array}{c}\hfill \frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}\hfill \\ \hfill \mathit{X}-E_J(\mathit{X}|T)\hfill \end{array}\mathbf{\right)}.$$

Step 2. We now project ℓ̇_g (a_g) onto the space generated by ℓ̇_h, using calculations similar to those in Step 1. Denote the least favorable direction as $a_{g,h}^{*}$. It follows that

$$\begin{array}{c}{E}_0{\{{\dot{\ell }}_g(a_g)-{\dot{\ell }}_h(a_{g,h})\}}^2\hfill \\ =E_0{\{D_1(\xi )a_g(T)-D_2(\eta )a_{g,h}(T)\}}^2\hfill \\ =C_1{E}_J{\{\frac{D_1(\xi )}{D_2(\eta )}{a}_g(T)-a_{g,h}(T)\}}^2\hfill \end{array}$$

is minimized at $a_{g,h}^{*}=E_J\{\frac{D_1(\xi )}{D_2(\eta )}{a}_g(T)|T\}=a_g(T)E_J\{\frac{D_1(\xi )}{D_2(\eta )}|T\}$. Then we have

$${\dot{\ell }}_g(a_g)-{\dot{\ell }}_h(a_{g,h}^{*})=D_2(\eta )a_g(T)[\frac{D_1(\xi )}{D_2(\eta )}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}|T\}].$$

Step 3. Next, we project the space generated by U* onto the space generated by ${\dot{\ell }}_g(a_g)-{\dot{\ell }}_h(a_{g,h}^{*})$. It is equivalent to finding ã_g(T) = (ã_g,β(T), ã_g,γ(T))^T, which minimizes each component of

$$\begin{array}{c}{E}_0{D}_2^2(\eta )\mathbf{\left(}\begin{array}{c}\hfill {\left[\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}-\{\frac{D_1(\xi )}{D_2(\eta )}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}|T\}\}{a}_{g,\beta }(T)\right]}^2\hfill \\ \hfill {[\mathit{X}-E_J\{X|T\}-\{\frac{D_1(\xi )}{D_2(\eta )}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}|T\}\}{a}_{g,\gamma }(T)]}^2\hfill \end{array}\mathbf{\right)}\hfill \\ =C_2{E}_H\mathbf{\left(}\begin{array}{c}\hfill {\left[{\delta }^{-1}\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}\}-a_{g,\beta }(T)\right]}^2\hfill \\ \hfill {[{\delta }^{-1}\{\mathit{X}-E_J(\mathit{X}|T)\}-a_{g,\gamma }(T)]}^2\hfill \end{array}\mathbf{\right)}.\hfill \end{array}$$

The solutions are

$$\begin{array}{cc}{\tilde{a}}_{g,\beta }(T)\hfill & =E_H\left[{\delta }^{-1}\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}\}|T\right]\hfill \\ \hfill & =E_H\left[\{\frac{D_1(\xi )}{D_2(\eta )}{\delta }^{-1}\mathit{X}\}|T\right]-E_H({\delta }^{-1}|T)E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\},\hfill \\ {\tilde{a}}_{g,\gamma }(T)\hfill & =E_H[\{{\delta }^{-1}\{\mathit{X}-E_J(\mathit{X}|T)\}\}|T]\hfill \\ \hfill & =E_H({\delta }^{-1}\mathit{X}|T)-E_H({\delta }^{-1}|T)E_J(\mathit{X}|T).\hfill \end{array}$$ (11)

Thus the efficient score function for (β, γ) is given by

$$\begin{array}{cc}{\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}\hfill & ={\mathit{U}}^{\mathbf{*}}-{\dot{\ell }}_g({\tilde{a}}_g)-{\dot{\ell }}_h({\tilde{a}}_{g,h}^{*})\hfill \\ \hfill & ={\mathit{U}}^{\mathbf{*}}-D_2(\eta )\delta {\tilde{a}}_g\hfill \\ \hfill & =D_2(\eta )\left(\begin{array}{c}\hfill \frac{D_1(\xi )}{D_2(\eta )}\mathit{X}-\delta E_H\{\{\frac{D_1(\xi )}{D_2(\eta )}{\delta }^{-1}\mathit{X}\}|T\}-(1-\delta E_H({\delta }^{-1}|T))E_J\{\frac{D_1(\xi )}{D_2(\eta )}\mathit{X}|T\}\hfill \\ \hfill \mathit{X}-\delta E_H({\delta }^{-1}\mathit{X}|T)-(1-\delta E_H({\delta }^{-1}|T))E_J(\mathit{X}|T)\hfill \end{array}\right),\hfill \end{array}$$

and the Fisher information matrix is I = E₀(ℓ̃_β,γℓ̃_β,γ^T).

4.2. Estimation of Variance

The variance-covariance matrix of (β̂_n, γ̂_n) is not expressed in nice forms that can be estimated from data. In similar settings, Huang (1999) suggested using the observed information matrix based on the log-likelihood function. Ma (2009) proposed a weighted bootstrap approach. In this article, we take the former approach.

Following the notation in Section 2, the log-likelihood function ℓ(β, γ, g, h; w) can be written as ℓ(β,γ, b, a; w), with g(t) replaced by G_m(t; b) and h(t) replaced by G_m(t; a). The observed joint information matrix I(β, γ, b, a) is given by

$$I(\mathit{\beta },\mathit{\gamma },\mathit{b},\mathit{a})=\mathbf{\left(}\begin{array}{cccc}\hfill -\frac{{\partial }^2\ell }{\partial {\mathit{\beta }}^2}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\beta }\partial \mathit{\gamma }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\beta }\partial \mathit{b}}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\beta }\partial \mathit{a}}\hfill \\ \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\gamma }\partial \mathit{\beta }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial {\mathit{\gamma }}^2}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\gamma }\partial \mathit{b}}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{\gamma }\partial \mathit{a}}\hfill \\ \hfill -\frac{{\partial }^2\ell }{\partial \mathit{b}\partial \mathit{\beta }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{b}\partial \mathit{\gamma }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial {\mathit{b}}^2}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{b}\partial \mathit{a}}\hfill \\ \hfill -\frac{{\partial }^2\ell }{\partial \mathit{a}\partial \mathit{\beta }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{a}\partial \mathit{\gamma }}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial \mathit{a}\partial \mathit{b}}\hfill & \hfill -\frac{{\partial }^2\ell }{\partial {\mathit{a}}^2}\hfill \end{array}\mathbf{\right)}.$$

The standard error of (β̂_n, γ̂_n, b̂, â) is approximately $I^{-1/2}({\widehat{\mathit{\beta }}}_{\mathit{n}},{\widehat{\mathit{\gamma }}}_{\mathit{n}},\widehat{\mathit{b}},\widehat{\mathit{a}})/\sqrt{n}$, and the standard error of (β̂_n, γ̂_n) can be obtained from part of this matrix. Though the inverse of the observed information matrix is often used to estimate the asymptotic variance of the finite-dimensional parameter in the parametric and semiparametric literature, it is difficult to prove asymptotic results in semiparametric cases, which was also pointed by Huang and Rossini (1997, p.962). In our empirical work, we note that the inverse of the observed information matrix provides a reasonable approximation to I^−1/2.

5 Simulation Studies

We conduct Monte Carlo simulations for two purposes. One is to verify that the estimated standard errors are reliable, and the other is to show the advantage of using a nonparametric component in Model (4).

5.1. Study I

We generate data from the following model

$$\text{log}(\lambda )={\beta }_1{X}_1+{\beta }_2{X}_2+g(T)\text{ and }\text{log}(\frac{p}{1-p})={\gamma }_1{X}_1+{\gamma }_2{X}_2+h(T),$$

where X₁, X₂ and T are independently drawn from the binomial distribution b(1, 0.5), the uniform distributions on [0, 2] and [0, 1], respectively, with the regression parameters β₁ = 0.3, β₂ = 0.2, γ₁ = −1.7 and γ₂ = 0.5, and the nonparametric components

$$g(t)=\mathit{\text{sin}}(\pi t),h(t)=3t^2+2.$$

In this model, the probability p ranges from 0.575 to 0.997 with mean 0.890. A sample size of n = 5000 is chosen so that it is similar to the case study in the next section. The following four working models (WM) are used to fit the data.

• WM1: both g(T) and h(T) are modeled nonparametrically;

• WM2: both g(T) and h(T) are modeled linearly;

• WM3: g(T) is modeled linearly but h(T) is modeled nonparametricaly;

• WM4: h(T) is modeled linearly but g(T) is modeled nonparametricaly.

When splines are used to approximate a nonparametric component in the model, we use uniform knots on [0,1]. The number of knots, m + 1, can be chosen adaptively by minimizing the Akaike information criterion (AIC) given by

$$\mathit{\text{AIC}}(m+1)=-2n{L}_n({\widehat{\theta }}_n,{\widehat{W}}_n)+2K(m+1),$$

where K(m + 1) is the number of parameters to be estimated in a given working model. To save time, we used AIC to choose m + 1 for only 100 Monte Carlo samples. Since the selected values were around 4 for approximating both functions g and h, we fix m + 1 = 4 for all the 1000 samples in the Monte Carlo study.

Table 1 presents the average of AIC(4), the estimated bias of (γ̂,β̂), and the Monte Carlo estimate of the standard deviations (SD(γ̂,β̂)) together with the average of the estimated standard errors based on the observed log-likelihood (SE(γ̂,β̂)). We learn that (i) the parametric estimators in WM2 – WM4 are clearly biased relative to the bias from WM1; (ii) the AIC values clearly favor WM1; (iii) there is good agreement between the Monte Carlo based standard error estimates and the estimated standard errors, indicating that the estimation method of Section 4.2 is reliable.

Table 1.

Study I on bias, SD (Monte Carlo estimate of the standard deviation), and SE (average standard error estimates based on information matrix).

	WM1	WM2	WM3	WM4
AIC	1980.91	2078.03	1998.19	2050.92
γ̂1(bias)	−0.0281	−0.1533	−0.1914	0.0120
(SD,SE)	(0.1088,0.1062)	(0.1007,0.1025)	(0.1040,0.1030)	(0.1081,0.1046)
γ̂2(bias)	−0.0168	−0.0977	−0.1052	−0.0035
(SD,SE)	(0.0801,0.0803)	(0.0775,0.0801)	(0.0784,0.0807)	(0.0816,0.0796)
β̂1(bias)	−0.0016	0.0435	−0.0165	0.0514
(SD,SE)	(0.0611,0.0568)	(0.0663,0.0548)	(0.0560,0.0539)	(0.0685,0.0572)
β̂2(bias)	−0.0002	0.0156	−0.0046	0.0173
(SD,SE)	(0.0418,0.0440)	(0.0476,0.0435)	(0.0414,0.0439)	(0.0481,0.0444)

Figures 1 – 2 show that the nonparametric estimators for either function h(t) or g(t) are able to capture their shapes reasonably well, but linear estimators cannot. This example demonstrates that the less-restrictive Working Models 2–4 would be clearly inferior to WM1.

Figure 1.

Plot of the Estimated h(t) in Study I.

Figure 2.

Plot of the Estimated g(t) in Study I.

5.2. Study II

Now we generate data in the same way except that the functions g and h are indeed linear: h(t) = 3t + 2 and g(t) = −2t + 1. The probability p ranges from 0.576 to 0.998 with mean 0.890 in this case. The results are summarized in Table 2 and Figures 3 – 4. Table 2 shows that even though the parametric estimates from working models 1, 3 and 4 are not as efficient as those under WM2, the loss is small. The empirical efficiency of WM1 relative to WM2 is over 97.5% for γ and β, where WM2 would be chosen most often by AIC.

Table 2.

Study II on bias, SD (Monte Carlo estimate of the standard deviation), and SE (average standard error estimates based on information matrix).

	WM1	WM2	WM3	WM4
AIC	1645.52	1641.70	1643.30	1643.64
γ̂1(bias)	−0.0156	−0.0041	−0.0071	−0.0212
(SD,SE)	(0.1359,0.1336)	(0.1173,0.1319)	(0.1189,0.1331)	(0.1362,0.1329)
γ̂2(bias)	−0.0125	−0.0094	−0.0103	−0.0152
(SD,SE)	(0.1002,0.1027)	(0.0955,0.1018)	(0.0963,0.1024)	(0.1007,0.1024)
β̂1(bias)	0.0273	0.0139	0.0283	0.0142
(SD,SE)	(0.0900,0.0799)	(0.0801,0.0792)	(0.0904,0.0797)	(0.0805,0.0795)
β̂2(bias)	0.0109	0.0064	0.0109	0.0065
(SD,SE)	(0.0619,0.0618)	(0.0594,0.0615)	(0.0619,0.0618)	(0.0594,0.0616)

Figure 3.

Plot of the Estimated h(t) in Study II.

Figure 4.

Plot of the Estimated g(t) in Study II.

6 An Example on Length of Sick Leaves

We use a data set from a public health survey conducted in Indonesia in 1997 (Fankenberg and Thomas, 2000) to illustrate the use of a semiparametric ZIP model. The response variable of interest is the number of days that people of working age missed their primary activities due to illness in a period of 4 weeks. The length of sick leaves is an important issue in public health policy studies. We focus on the subjects of working age (18–60 years) in the analysis, leading to a sample of size n = 5700, of which 5330 were zero counts. The factors of interest include gender (X₁ = 1 for female and 0 otherwise), age (X₂ ranged from 18 to 60 years old), household hygiene index (X₃ ranged from 0 to 5 from best to worst), and per capita annual household income (T ranged from 0 to 15555.6 in thousand Indonesian Rupiah). Lam, Xue and Cheung (2006) used Model (3) where the age effect enters the Poisson model nonparametrically.

In this article, we fit the doubly semiparametric ZIP model where per capita annual household income (T) entered the model nonparametrically. This choice was made by the AIC criterion after considering various working models as in the previous section with either age or household income as T.

For computational stability, we map all the variables X1 – X3 and T to the interval [0, 1] through a linear transformation. Because the income variable is highly left-skewed in the data(see Figure 5), we do not use uniformly spaced knots as we did in simulation studies, but choose to use a rather extensive knot search based on AIC for given number of knots m + 1 between 2 and 4. Of course, both methods for knots selection do not affect the methodology and theoretical results. The fitted semiparametric ZIP model has knots (0, 0.07, 1) for h and (0, 0.03, 0.04, 1) for g. Note that several hundreds of observations fall into the “small” interval of T ∈ (0.03, 0.04) in this data set. The estimation results are summarized in Table 3 and Figures 6 – 7.

Figure 5.

Plot of Quantiles of the Per Capita Annual Household Income.

Table 3.

Results of the analyses based on the survey data; the AIC criterion favors Model WM1.

	WM1	WM2	WM3	WM4
AIC	946.20	955.22	954.71	946.76
γ̂1(SE)	−1.6851(0.1055)	−1.7040(0.1052)	−1.6863(0.1053)	−1.7030(0.1053)
γ̂2(SE)	−0.3669(0.1471)	−0.3888(0.1466)	−0.3668(0.1469)	−0.3892(0.1467)
γ̂3(SE)	−0.4989(0.2307)	−0.5252(0.2308)	−0.4998(0.2306)	−0.5245(0.2309)
β̂1(SE)	0.3179(0.0530)	0.3240(0.0527)	0.3239(0.0528)	0.3180(0.0529)
β̂2(SE)	0.1994(0.0772)	0.2230(0.0768)	0.2231(0.0769)	0.1991(0.0772)
β̂3(SE)	0.2816(0.1022)	0.2733(0.1025)	0.2731(0.1024)	0.2819(0.1022)

Figure 6.

Plot of the Estimated h(t) for the Public Health Data; Model WM1 was chosen for our analysis, but Model WM2 is shown for comparison.

Figure 7.

Plot of the Estimated g(t) for the Public Health Data.

Table 3 suggests that the covariates gender, hygiene condition and age are all significant at the 5% level. For the gender and hygiene effect, the results are consistent with those of Lam, Xue and Cheung (2006). The age effect in our study appears more pronounced than shown in Lam, Xue and Cheung (2006). The per capita annual household income was found to be insignificant by Lam, Xue and Cheung (2006), but when it is modeled nonparametrically, we note from Figure 6 that the probability of zero sick leave day increases with income between 0 and 1089 (thousand Rupiah) and then decreases after that. From Figure 7, the mean sick leave days λ from the high-risk group decreases with income when it is below 467 (thousand Rupiah), but shows an up-tick after that, until it shows a downward trend again for income above 622 (thousand Rupiah). Note that 11% of the subjects in the data set have per capita household income between 467 and 622, so the direction change of g(t) in that region suggests that the relationship in g, after adjusting for the other three covariares, is quite complicated. Without adjusting for the other covariates, this slope reversal in g did not occur, suggesting that a more careful analysis for this sub-group is needed. At this point, we cannot claim that the doubly semi-parametric ZIP model used here is adequate in this analysis, but the difference between the results obtained from a parametric ZIP model and a semiparametric one are often indicative of the need for further studies.

7 Conclusion

In this article, we study the sieve estimators for nonparametric functions g and h in doubly semiparametric zero-inflated models. For simplicity and understanding easily, we use piecewise linear functions in the estimates, but the theories derived in this article hold true for other B-splines or polynomial splines. And our results can be easily extended to the case that both g and h are additive functions. Moreover, the same estimation method can be applied to other exponential dispersion family distributions, such as binomial and negative binomial, and the asymptotic studies are similar. Our main contribution is to derive the efficient scores for the sieve maximum likelihood estimation when two nonparametric functions are being estimated in different parts of the model, which could not be handled by the simpler techniques used in Lam, Xue and Cheung (2006). The doubly semiparametric ZIP model offers more flexibility than linear or semiparametric ZIP models studied earlier, and this paper provides a solid mathematical theory for efficient estimation. One important point that deserves serious consideration is the identifiability problem. Though under some assumptions, this doubly semiparametric ZIP model is identifiable, applications of this complex type of mixture model are restricted to problems in which there is strong evidence for the existence of the cured population.

Appendix: Proof of Theorems

The assumptions C1–C3 needed for the results in Sections 2 and 3 are listed below:

• C1. The variable X has a bounded support, and T ∈ [0, 1]. The true parameters β₀ ∈ A₁, γ₀ ∈ A₂, where A₁ and A₂ are compact sets in R^d₁+1.

• C2. g₀, h₀ ∈ C^r[0, 1] for r = 1 or 2.

• C3. max_1≤j≤m(t_j − t_j−1) ≤ Cn^−k for some constant C and 0 < k < 1.

The conditions A1–A4 needed for the theorems in Section 3 are listed below:

• A1. θ₀ is an interior point of Θ. In other words, (β₀, γ₀) is an interior point of A₁ * A₂, and m₀ < g₀(t) < M₀, and m₁ < h₀(t) < M₁ for all t, where the constants m₀,M₀,m₁ and M₁ are given in the definitions B_g and B_h in Section 2.

• A2. The joint density function φ(x, t) of (X, T) is second order continuously differentiable in t with a bounded derivative.

• A3. The partition of [0, 1] is such that $\underset{j}{\text{min}}(t_j-t_{j-1})=O(n^{-k\prime })\text{with}\frac{1}{5}<k<\frac{1}{3}\text{for}k\le k\prime <\frac{1-k}{2},\text{or}\frac{1}{8}<k\le \frac{1}{5}\text{for}k\le k\prime <2k.$.

• A4. The functions g₀ and h₀ are second order continuously differentiable.

Let N(ε, S,L_∞) be the covering number of the class S, as given in Pollard (1984, p. 25). Then we have the following lemma.

Lemma 1 N(ε,B_g,n, L_∞) ≤ (6M₁/ε)^m+1, N(ε,B_h,n,L_∞) ≤ (6M₂/ε)^m+1, and N(ε,Λ_n,L_∞) ≤ K(1/ε)^2(m+d₁), where Λ_n = {ℓ(θ, .) : θ ∈ Θ_n}, and K is a constant.

Lemma 1 and the proof of the first part of Theorem 1 follow the arguments in Xue, Lam and Li(2004), the proof of the second part of Theorem 1 follows standard lines by applying Theorem 3.4.1 of van der Vaart and Wellner(1996), and therefore the proofs are skipped here.

Proof of Theorem 3: Some definitions are needed to prove this theorem. Rewrite the score functions of ζ ≐ (β^T, γ^T)^T, g and h in (8)–(11) as ℓ̇₁ (θ,W), ℓ̇₂ (θ,W)[a₂] and ℓ̇₃(θ,W)[a₃]. Denote

$$S_{1n}({\widehat{\mathit{\theta }}}_n)=P_n{\dot{\mathit{\ell }}}_1({\widehat{\mathit{\theta }}}_n,\mathit{W})\text{ and }{S}_{\mathit{\text{jn}}}({\widehat{\mathit{\theta }}}_n)[a_j]=P_n{\dot{\mathit{\ell }}}_j({\widehat{\mathit{\theta }}}_n,\mathit{W})[a_j],$$

where j runs from 2 to 3. Let S₁ and S_j be the limiting versions of S_1n and S_jn; that is S₁(θ) = P ℓ̇₁(θ,W) and S_j(θ) = P ℓ̇_j(θ,W)[a_j], where P = P_θ0. Define

$$\begin{array}{c}\hfill {\dot{S}}_{11}(\mathit{\theta })=-P{\dot{\mathit{\ell }}}_1(\mathit{\theta },\mathit{W}){\dot{\mathit{\ell }}}_1^T(\mathit{\theta },\mathit{W}),\hfill \\ \hfill {\dot{S}}_{1j}(\mathit{\theta })[a_j]={\dot{S}}_{j1}^T(\mathit{\theta })[a_j]=-P{\dot{\mathit{\ell }}}_1(\mathit{\theta },\mathit{W}){\dot{\ell }}_j^T(\mathit{\theta },\mathit{W})[a_j],\hfill \end{array}$$

and

$${\dot{S}}_{\mathit{\text{jk}}}(\mathit{\theta })[a_j,a_k]={\dot{S}}_{\mathit{\text{kj}}}^T(\mathit{\theta })[a_k,a_j]=-P{\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[a_j]{\dot{\ell }}_k^T(\mathit{\theta },\mathit{W})[a_k],$$

where both j and k run from 2 to 3. Furthermore, for a_j = (a_j1, ⋯, a_jd)^T and a_k = (a_k1, ⋯, a_kd)^T, where a_ji ∈ A_g for j = 2 and a_ji ∈ A_h for j = 3, i = 1, ⋯, d, and a_ki similarly, denote

$$\begin{array}{c}\hfill {\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[{\mathit{a}}_j]={({\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[a_{j1}],\cdots ,{\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[a_{\mathit{\text{jd}}}])}^T,\hfill \\ \hfill {\dot{S}}_j(\mathit{\theta })[{\mathit{a}}_j]=P{\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[{\mathit{a}}_j],{\dot{S}}_{\mathit{\text{jn}}}(\mathit{\theta })[{\mathit{a}}_j]=P_n{\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[{\mathit{a}}_j],\hfill \\ \hfill {\dot{S}}_{1j}(\mathit{\theta })[{\mathit{a}}_j]={\dot{S}}_{j1}^T(\mathit{\theta })[{\mathit{a}}_j]=-P{\dot{\mathit{\ell }}}_1(\mathit{\theta },\mathit{W}){\dot{\mathit{\ell }}}_j^T(\mathit{\theta },\mathit{W})[{\mathit{a}}_j],\hfill \end{array}$$

and

$${\dot{S}}_{\mathit{\text{jk}}}(\mathit{\theta })[{\mathit{a}}_j,{\mathit{a}}_k]={\dot{S}}_{\mathit{\text{kj}}}^T(\mathit{\theta })[{\mathit{a}}_k,{\mathit{a}}_j]=-P{\dot{\mathit{\ell }}}_j(\mathit{\theta },\mathit{W})[{\mathit{a}}_j]{\dot{\mathit{\ell }}}_k^T(\mathit{\theta },\mathit{W})[{\mathit{a}}_k].$$

We can obtain the following results B1–B4.

• B1. From Theorem 1, we have that ‖ζ̂_n − ζ₀| = o_p(1), ‖ĝ_n − g₀‖ = O_p(n^−α) and ‖ĥ_n − h₀‖ = O_p(n^−α), where O_p(n^−α) is just the rate of convergence in Theorem 1.

• B2. By selecting the least favorable direction as ${\mathit{a}}_{\mathit{g}}^{\mathbf{*}}\text{ and }{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}$ in Section 4, we have

  $${\dot{S}}_{1j}({\mathit{\theta }}_0)[a_j]={\dot{S}}_{2j}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}},a_j]={\dot{S}}_{3j}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}},a_j]=0,$$

  for all a₂ ∈ A_g and a₃ ∈ A_h, where j runs from 2 to 3. Furthermore,

  $${\dot{S}}_{11}({\mathit{\theta }}_0)-{\dot{S}}_{21}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-{\dot{S}}_{31}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}]$$

  is nonsingular.
• B3. For any δ_n ↓ 0 and C₁, C₂ > 0, set 𝒜 = {θ = (ζ^T, g, h)^T : ‖ζ − ζ₀‖ ≤ δ_n, ‖g − g₀‖₂ ≤ C₁n^−α, ‖h − h₀‖₂ ≤ C₂n^−α}, similar to C3 of Theorem 4 in Xue, Lam and Li(2004), it follows that

  $$\underset{𝒜}{\text{sup}}|\sqrt{n}(S_{1n}-S_1)(\mathit{\theta })-\sqrt{n}(S_{1n}-S_1)({\mathit{\theta }}_0)|=o_p(1),$$

  and

  $$\underset{𝒜}{\text{sup}}|\sqrt{n}(S_{\mathit{\text{jn}}}-S_j)(\mathit{\theta })[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]-\sqrt{n}(S_{\mathit{\text{jn}}}-S_j)({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]|=o_p(1),$$

  where j runs from 2 to 3.
• B4. Similar to C4 of Theorem 4 in Xue, Lam and Li (2004), for θ ∈ 𝒜, it follows that

  $$|S_1(\mathit{\theta })-S_1({\mathit{\theta }}_0)-{\dot{S}}_{11}({\mathit{\theta }}_0)(\mathit{\zeta }-{\mathit{\zeta }}_0)-{\dot{S}}_{12}({\mathit{\theta }}_0)[g-g_0]-{\dot{S}}_{13}({\mathit{\theta }}_0)[h-h_0]|=o(\Vert \mathit{\zeta }-{\mathit{\zeta }}_0\Vert )+O({\Vert g-g_0\Vert }_2^2)+O({\Vert h-h_0\Vert }_2^2),$$

  and

  $$|S_j(\mathit{\theta })[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]-S_j({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]-({\dot{S}}_{j1}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}])(\mathit{\zeta }-{\mathit{\zeta }}_0)-{\dot{S}}_{j2}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}},g-g_0]-{\dot{S}}_{j3}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}},h-h_0]|=o(\Vert \mathit{\zeta }-{\mathit{\zeta }}_0\Vert )+O({\Vert g-g_0\Vert }_2^2)+O({\Vert h-h_0\Vert }_2^2),$$

  where j runs from 2 to 3.

Since $S_{1n}({\widehat{\mathit{\theta }}}_n)=o_p(n^{-\frac{1}{2}}),S_{\mathit{\text{jn}}}({\widehat{\mathit{\theta }}}_n)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]=o_p(n^{-\frac{1}{2}}),S_1({\mathit{\theta }}_0)=0,\text{ and }{S}_j({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]=0$ for j = 2 or 3, we have, by B1 and B3,

$$\sqrt{n}{S}_{\mathrm{l}}({\widehat{\mathit{\theta }}}_n)+\sqrt{n}{S}_{1n}({\mathit{\theta }}_0)=o_p(1)\text{ and }\sqrt{n}{S}_j({\widehat{\mathit{\theta }}}_n)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]+\sqrt{n}{S}_{\mathit{\text{jn}}}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{\mathbf{*}}]=o_p(1).$$

Together with B4, the above results imply that

$${\dot{S}}_{11}({\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0)+{\dot{S}}_{12}[{\widehat{g}}_n-g_0]+{\dot{S}}_{13}({\widehat{h}}_n-h_0)+(o(\Vert {\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0\Vert )+O({\Vert {\widehat{g}}_n-g_0\Vert }_2^2)+O({\Vert {\widehat{h}}_n-h_0\Vert }_2^2))+S_{1n}({\mathit{\theta }}_0)=o_p(n^{-\frac{1}{2}}),$$ (13)

and

$${\dot{S}}_{j1}[{\mathit{a}}_{\mathit{j}}^{*}]({\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0)+{\dot{S}}_{22}[{\mathit{a}}_{\mathit{j}}^{*},{\widehat{g}}_n-g_0]+{\dot{S}}_{j3}({\mathit{a}}_{\mathit{j}}^{*},{\widehat{h}}_n-h_0)+(o(\Vert {\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0\Vert )+O({\Vert {\widehat{g}}_n-g_0\Vert }_2^2)+O({\Vert {\widehat{h}}_n-h_0\Vert }_2^2))+S_{2n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{j}}^{*}]=o_p(n^{-\frac{1}{2}}).$$ (14)

By Theorem 1, $\sqrt{n}O({\Vert {\widehat{g}}_n-g_0\Vert }_2^2)=o_p(1)\text{and}\sqrt{n}O({\Vert {\widehat{h}}_n-h_0\Vert }_2^2)=o_p(1)$. Taking the difference between Equation (13) and Equations (14) for j = 2 and 3 together yields

$$({\dot{S}}_{11}-{\dot{S}}_{21}[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-{\dot{S}}_{31}[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}])({\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0)+o(\Vert {\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0\Vert )=-(S_{1n}({\mathit{\theta }}_0)-S_{2n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-S_{3n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}])+o_p(n^{-\frac{1}{2}}).$$

It follows that

$$\sqrt{n}({\dot{S}}_{11}-{\dot{S}}_{21}[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-{\dot{S}}_{31}[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}])({\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0)=-\sqrt{n}(S_{1n}({\mathit{\theta }}_0)-S_{2n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-S_{3n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}])+o_p(1).$$

From Section 4, the efficient score for (β, γ) is

$${\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}(\mathit{\theta },\mathit{W})={\dot{\mathit{\ell }}}_1(\mathit{\theta },\mathit{W})-{\dot{\mathit{\ell }}}_2(\mathit{\theta },\mathit{W})[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-{\dot{\mathit{\ell }}}_3(\mathit{\theta },\mathit{W})[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}],$$

and its information is

$$\mathit{I}({\mathit{\theta }}_0)=-{\dot{S}}_{11}({\mathit{\theta }}_0)+{\dot{S}}_{21}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]+{\dot{S}}_{31}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}]=E({\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}{\tilde{\ell }}_{\mathit{\beta },\mathit{\gamma }}^{\mathit{T}}).$$

Since

$$S_{1n}({\mathit{\theta }}_0)-S_{2n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{g}}^{\mathbf{*}}]-S_{3n}({\mathit{\theta }}_0)[{\mathit{a}}_{\mathit{h}}^{\mathbf{*}}]=P_n{\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}({\mathit{\theta }}_0,\mathit{W}),$$

we have

$$\sqrt{n}{P}_n{\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}({\mathit{\theta }}_0,\mathit{W})\stackrel{d}{\to }N(0,\mathit{I}({\mathit{\theta }}_0)).$$

Thus $\sqrt{n}({\widehat{\mathit{\zeta }}}_n-{\mathit{\zeta }}_0)={\mathit{I}}^{-1}({\mathit{\theta }}_{\mathbf{0}})\sqrt{n}{P}_n{\tilde{\mathit{\ell }}}_{\mathit{\beta },\mathit{\gamma }}({\mathit{\theta }}_0,\mathit{W})+o_p(1)\stackrel{d}{\to }N(0,{\mathit{I}}^{-1}({\mathit{\theta }}_0)).$.