A Z-theorem with Estimated Nuisance Parameters and Correction Note for ‘Weighted Likelihood for Semiparametric Models and Two-phase Stratified Samples, with Application to Cox Regression’

Abstract

We state and prove a limit theorem for estimators of a general, possibly infinite dimensional parameter based on unbiased estimating equations containing estimated nuisance parameters. The theorem corrects a gap in the proof of one of the assertions of our paper ‘Weighted likelihood for semiparametric models and two-phase stratified samples, with application to Cox regression’.

1. Introduction

Breslow & Wellner (2007) cited a theorem of Pierce (1982) in deriving the asymptotic distribution of weighted likelihood estimators for parameters in semiparametric models fitted to two-phase stratified samples when sampling weights were estimated from the data. Li (2007, personal communication) noticed that one of Pierce’s two key hypotheses had in fact not been established by us. In this note we develop a Z-theorem with estimated nuisance parameters that applies to infinite dimensional parameters and allows us to complete our earlier proof under a slight strengthening of our previous hypotheses. The derivations use empirical process techniques developed in van der Vaart & Wellner (1996) and related articles. In order to keep the exposition as short as possible, we assume familiarity with the notation and results in section 3.3 of van der Vaart & Wellner (1996) and section 6 of Breslow & Wellner (2007).

2. A Z-theorem with estimated nuisance parameters

Following van der Vaart & Wellner (1996, section 3.3), define random and fixed maps Ψ_n(θ, α): ℋ → ℝ, Ψ(θ, α): ℋ → ℝ for some index set ℋ with Ψ_n(θ, α), Ψ(θ, α) ∈ ℓ^∞(ℋ). In most applications, including that in section 3, Ψ_n(θ, α)h = ℙ_nψ_θ,α,h and Ψ(θ, α)h = Pψ_θ,α,h for given measurable functions ψ_θ,α,h indexed by θ ∈ Θ, α ∈ and h ∈ ℋ. We do not insist on this in the general theorem, however.

Suppose that Ψ(θ₀, α₀) = 0. Here α is to be regarded as a nuisance parameter; in the application, α is a finite-dimensional parameter while θ = (ν, η) where ν is finite-dimensional and η is infinite-dimensional. Suppose we have available estimators α̂ = α̂_n of α, and then consider estimators θ̂_n of θ satisfying

$$\underset{h\in \mathcal{H}}{sup}\mid {\mathrm{\Psi }}_n({\widehat{\theta }}_n,{\widehat{\alpha }}_n)h\mid =o_p^{\ast }(n^{-1/2}).$$ (1)

We would like to establish limit theorems for $\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)$ which are similar to those in the standard Z-theorem of van der Vaart (1995); see also theorem 3.3.1, p. 310, of van der Vaart & Wellner (1996). As argued in van der Vaart & Wellner (2007, p. 235), we can derive limit distributions of θ̂_n based on {Ψ_n(θ, α̂): θ ∈ Θ} from the corresponding theory for {Ψ_n(θ, α₀): θ ∈ Θ}, if we know that $\sqrt{n}({\widehat{\alpha }}_n-{\alpha }_0)=O_p(1)$ and if we show that

$$\underset{\theta \in \mathrm{\Theta }}{sup}\mid \mid \sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })(\theta ,{\widehat{\alpha }}_n)-\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })(\theta ,{\alpha }_0)\mid {\mid }_{\mathcal{H}}=o_p^{\ast }(1).$$ (2)

An alternative goal would be to relate the estimators θ̂_n to estimators ${\widehat{\theta }}_n^0$ that satisfy ${sup}_{h\in \mathcal{H}}\mid {\mathrm{\Psi }}_n({\widehat{\theta }}_n^0,{\alpha }_0)h\mid =o_p^{\ast }(n^{-1/2})$. This is accomplished in the third part of the following theorem which generalizes theorem 5.31, p. 60, of van der Vaart (1998); see also theorem 6.18, p. 407, of van der Vaart (2002).

Theorem 1

Suppose that θ̂_n satisfies (1), that θ ↦ {Ψ(θ, α)h: h ∈ ℋ} is uniformly Fréchet-differentiable in a neighbourhood of α₀ with derivative maps Ψ̇ (θ₀, α) satisfying Ψ̇(θ₀, α) → Ψ̇(θ₀, α₀) ≡ Ψ̇₀ as α → α₀ with Ψ̇₀ continuously invertible. Suppose, moreover, that ${\mathbb{Z}}_n\equiv \sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)⇝{\mathbb{Z}}_0$ in ℓ^∞ (ℋ), that (2) holds and that

$${\Vert \sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\widehat{\theta }}_n,{\alpha }_0)-\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)\Vert }_{\mathcal{H}}=o_p^{\ast }(1+\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid ).$$ (3)

1. If $\mid \mid \sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))\mid {\mid }_{\mathcal{H}}=O_p^{\ast }(1)$, then $\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid =O_p^{\ast }(1)$ and

   $$\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}[\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))]+o_p^{\ast }(1).$$ (4)

2. If the map α↦ {Ψ(θ₀, α)h: h ∈ ℋ} is Fréchet-differentiable at α₀ with derivative map Ψ̇_α and $\sqrt{n}({\widehat{\alpha }}_n-{\alpha }_0)={\mathbb{G}}_n\varphi +o_p^{\ast }(1)$ satisfies (ℤ_n, $\sqrt{n}({\widehat{\alpha }}_n-{\alpha }_0)$) ⇝ (ℤ₀, ϕ), then

   $$\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)⇝-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}({\mathbb{Z}}_0+{\stackrel{.}{\mathrm{\Psi }}}_{\alpha }\mathbb{G}\varphi ).$$ (5)

3. Under the same hypotheses as in (i)

   $$\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=\sqrt{n}({\widehat{\theta }}_n^0-{\theta }_0)-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))+o_p^{\ast }(1).$$ (6)

4. Under the same hypotheses as in (ii)

   $$\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=\sqrt{n}({\widehat{\theta }}_n^0-{\theta }_0)-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}{\stackrel{.}{\mathrm{\Psi }}}_{\alpha }\sqrt{n}({\widehat{\alpha }}_n-{\alpha }_0)+o_p^{\ast }(1).$$ (7)

Proof

By the definition of θ̂_n,

$$\begin{array}{l}\sqrt{n}\left(\mathrm{\Psi }({\widehat{\theta }}_n,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)\right)\\ =\sqrt{n}\left(\mathrm{\Psi }({\widehat{\theta }}_n,{\widehat{\alpha }}_n)-{\mathrm{\Psi }}_n({\widehat{\theta }}_n,{\widehat{\alpha }}_n)\right)-\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_n))+o_p^{\ast }(1)\\ =-\sqrt{n}\left({\mathrm{\Psi }}_n({\widehat{\theta }}_n,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\widehat{\theta }}_n,{\widehat{\alpha }}_n)\right)+\sqrt{n}({\mathrm{\Psi }}_n({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n))\\ -\sqrt{n}\left({\mathrm{\Psi }}_n({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)\right)-\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))+o_p^{\ast }(1)\\ =-\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\widehat{\theta }}_n,{\alpha }_0)+\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)\\ -\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)-\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))+o_p^{\ast }(1)\text{by}(2)\\ =o_p^{\ast }(1+\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid )-\sqrt{n}({\mathrm{\Psi }}_n-\mathrm{\Psi })({\theta }_0,{\alpha }_0)-\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))+o_p^{\ast }(1)\end{array}$$ (8)

by using (3) in the last line. By uniform differentiability of θ ↦ {Ψ(θ, α)h: h ∈ ℋ} and uniform non-singularity of Ψ̇(θ₀, α), it follows that there is a constant c > 0 such that, for all (θ, α) in a sufficiently small neighbourhood of (θ₀, α₀), ||Ψ(θ, α) − Ψ(θ₀, α)|| ≥ c||θ − θ₀||. Combining this with (8) yields

$$c\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid \le o_p^{\ast }(1+\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid )+\mid \mid {\mathbb{Z}}_n\mid \mid +\mid \mid \sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))\mid \mid$$

and hence that $\sqrt{n}\mid \mid {\widehat{\theta }}_n-{\theta }_0\mid \mid =O_p^{\ast }(1)$ if the last term in the preceding display is O_p(1). Now

$$\begin{array}{l}\sqrt{n}({\widehat{\theta }}_n-{\theta }_0)=-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}[{\mathbb{Z}}_n+\sqrt{n}(\mathrm{\Psi }({\theta }_0,{\widehat{\alpha }}_n)-\mathrm{\Psi }({\theta }_0,{\alpha }_0))]+o_p^{\ast }(1)\\ =-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}({\mathbb{Z}}_n+{\stackrel{.}{\mathrm{\Psi }}}_{\alpha }{\mathbb{G}}_n\varphi )+o_p^{\ast }(1)\end{array}$$

where in the first equation we have again used the uniform differentiability hypothesis and in the second the hypotheses of part (ii) of the theorem. As this converges to the claimed limit, this proves (i) and (ii). To prove (iii) and (iv), note that the standard Z-theorem yields $\sqrt{n}({\widehat{\theta }}_n^0-{\theta }_0)=-{\stackrel{.}{\mathrm{\Psi }}}_0^{-1}{\mathbb{Z}}_n+o_p^{\ast }(1)$. The claimed results follow by combining each line in the last display with (4).

Remark

Under the hypotheses of theorem 1, theorem 2.21 of Kato (1976, p. 205) implies that the derivative maps Ψ̇(θ₀, α) are continuously invertible for α in a neighbourhood of α₀.

3. Completion of the proof of Breslow & Wellner (2007)

In Breslow & Wellner (2007), = {P_{θ, η}: θ ∈ Θ, η ∈ Ξ} is a semiparametric model that satisfies five assumptions A1–A5. Here we slightly strengthen A1, which had already strengthened the hypotheses of van der Vaart (1998, theorem 25.90), to

A1^* for (θ, η) in a δ-neighbourhood of (θ₀, η₀) the functions ℓ̇ _{θ, η} and {B_{θ, η}h − P_{θ, η}B_{θ, η}h, h ∈ ℋ} are contained in a P₀-Donsker class ℱ and have square-integrable envelope functions F₁ and F₂ respectively.

We also strengthen A3 to

A3^* A3 holds and moreover the derivative maps Ψ̇₀ = (Ψ̇₁₁, Ψ̇₁₂, Ψ̇₂₁, Ψ̇₂₂) have representations

$${\stackrel{.}{\mathrm{\Psi }}}_{ij}({\theta }_0,{\eta }_0)h=P_0({\stackrel{.}{\psi }}_{ij,{\theta }_0,{\eta }_0,h}),i,j\in \{1,2\}$$

in terms of L₂(P₀) derivatives of ψ_{1, θ, η, h} = ℓ̇ _{θ, η} and ψ_{2, θ, η, h} = B_{θ, η}h − P_{θ, η}B_{θ, η}h, i.e.

$$\begin{array}{l}\underset{h\in \mathcal{H}}{sup}{\left\{P_0{\left({\psi }_{i,\theta ,{\eta }_0,h}-{\psi }_{i,{\theta }_0,{\eta }_0,h}-{\stackrel{.}{\psi }}_{i1,{\theta }_0,{\eta }_0,h}(\theta -{\theta }_0)\right)}^2\right\}}^{1/2}=o(\mid \mid \theta -{\theta }_0\mid \mid ),\\ \underset{h\in \mathcal{H}}{sup}{\left\{P_0{\left({\psi }_{i,{\theta }_0,\eta ,h}-{\psi }_{i,{\theta }_0,{\eta }_0,h}-{\stackrel{.}{\psi }}_{i2,{\theta }_0,{\eta }_0,h}(\eta -{\eta }_0)\right)}^2\right\}}^{1/2}=o(\mid \mid \eta -{\eta }_0\mid \mid ),\end{array}$$

for i = 1, 2.

Breslow & Wellner (2007) showed in (25) on p. 92 that for π= π₀

$$\sqrt{N}\left(\begin{array}{c}{\widehat{\theta }}_N({\alpha }_0)-{\theta }_0\\ {\widehat{\alpha }}_N-{\alpha }_0\end{array}\right)=\sqrt{N}\left(\begin{array}{c}{\mathbb{P}}_N^{\pi }{\stackrel{\sim }{\ell }}_0\\ {\mathbb{Q}}_N{\stackrel{\sim }{\ell }}_0^{\alpha }\end{array}\right)+o_p(1)$$ (9)

and thus that the quantity on the LHS has an asymptotic N(0, V) distribution where

$$\begin{array}{l}V=\left(\begin{array}{ll}{V}_{11}\hfill & V_{12}\hfill \\ V_{21}\hfill & V_{22}\hfill \end{array}\right),V_{11}={\stackrel{\sim }{I}}_0^{-1}+{\stackrel{\sim }{P}}_0\left(\frac{1-{\pi }_0}{{\pi }_0}{\stackrel{\sim }{\ell }}_0^{\otimes 2}\right),\\ V_{22}={\left({\stackrel{\sim }{P}}_0{1}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^{\otimes 2}}{{\pi }_0(1-{\pi }_0)}\right)}^{-1},V_{12}=V_{21}^T={\stackrel{\sim }{P}}_0\left(\frac{\xi }{{\pi }_0}{\stackrel{\sim }{\ell }}_0{\stackrel{\sim }{\ell }}_0^{\alpha }\right)={\stackrel{\sim }{P}}_0\left({\stackrel{\sim }{\ell }}_0{\stackrel{\sim }{\ell }}_0^{\alpha }\right).\end{array}$$

Thus Pierce’s (1982) first hypothesis (1.1) is satisfied with θ̂(α₀) − θ₀ the statistic of interest, T_n in his notation, and α the estimated nuisance parameter, Pierce’s (1982)λ.

Pierce’s (1982) second hypothesis (1.2) is that with T̂_n = T_n(λ̂_n)

$$\sqrt{n}{\widehat{T}}_n=\sqrt{n}{T}_n+B\sqrt{n}({\widehat{\lambda }}_n-\lambda )+o_p(1)$$ (10)

for some matrix B. A further hypothesis is that λ̂_n is efficient; i.e. $V_{22}=I_{\lambda }^{-1}$. Then Pierce shows that $\sqrt{n}{\widehat{T}}_n⇝N(0,V_{11}-B{V}_{22}{B}^T)$. Breslow & Wellner (2007) also showed in (26) on p. 92 that

$$\sqrt{N}({\mathbb{P}}_N^{\widehat{\pi }}-{\mathbb{P}}_N^{{\pi }_0}){\stackrel{\sim }{\ell }}_0=-{\stackrel{\sim }{P}}_0\left({\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{\sim }{\ell }}_0{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right)\sqrt{N}({\widehat{\alpha }}_N-{\alpha }_0)+o_p(1).$$ (11)

However, as pointed out by Li, this does not yet prove that

$$\sqrt{N}({\widehat{\theta }}_N(\widehat{\alpha })-{\theta }_0)=\sqrt{N}({\widehat{\theta }}_N({\alpha }_0)-{\theta }_0)+B\sqrt{N}({\widehat{\alpha }}_N-{\alpha }_0)+o_p(1)$$ (12)

for some matrix B as is needed to verify (10). This result does, however, follow from part (iv) of theorem 1.

As in section 6 of Breslow & Wellner (2007), suppose that α̂ = α̂_N denotes the maximum likelihood estimator of parameters in the model π_α(ν) for the sampling probabilities and that θ̂_N ≡ θ̂_N (α̂), η̂_N ≡ η̂_N (α̂) solve ${\mathbb{P}}_N^{\widehat{\pi }}{\stackrel{.}{\ell }}_{\theta ,\eta }=0$ and ${\mathbb{P}}_N^{\widehat{\pi }}{B}_{\theta ,\eta }h-P_{\theta ,\eta }{B}_{\theta ,\eta }h=0$ for all h ∈ ℋ where

$${\mathbb{P}}_N^{\widehat{\pi }}=\frac{1}{N}\sum _{i=1}^N\frac{{\xi }_i}{{\widehat{\pi }}_i}{\delta }_{X_i},$$

and where π̂_i ≡ π_α̂(V_i), i = 1, …, N.

Theorem 2

Suppose the semiparametric model satisfies A1^* and A3^* above and A2, A4 and A5 of Breslow & Wellner (2007) and that the model π_α (V) for the conditional distribution of ξ given X, V satisfies the hypotheses of theorem 5.39 of van der Vaart (1998). Suppose moreover that π_α satisfies (42) of Breslow & Wellner (2007):

$$\mid \frac{1}{{\pi }_{\alpha }(v)}-\frac{1}{{\pi }_{{\alpha }_0}(v)}-\frac{-{\stackrel{.}{\pi }}_0^T(v)}{{\pi }_0^2(v)}(\alpha -{\alpha }_0)\mid \le \psi (v)\mid \alpha -{\alpha }_0{\mid }^{1+\zeta }$$ (13)

for α in a neighbourhood of α₀ where ξ > 0 and ψ satisfies Eψ²(V) < ∞. Then

$$\sqrt{N}({\widehat{\theta }}_N(\widehat{\alpha })-{\theta }_0)⇝Z\sim N_p(0,\mathrm{\sum })$$

where, as in (27) of Breslow & Wellner (2007),

$$\mathrm{\sum }=\text{Var}\left(\frac{\xi }{{\pi }_0}{\stackrel{\sim }{\ell }}_0\right)-{\stackrel{\sim }{P}}_0{1}_{{\mathcal{V}}_0^c}\frac{{\stackrel{\sim }{\ell }}_0{\stackrel{.}{\pi }}_0^T}{{\pi }_0}{\left({\stackrel{\sim }{P}}_0{1}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^{\otimes 2}}{{\pi }_0(1-{\pi }_0)}\right)}^{-1}{\stackrel{\sim }{P}}_0{1}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0{\stackrel{\sim }{\ell }}_0^T}{{\pi }_0}.$$

Proof

We use theorem 1 with θ there replaced by (θ, η). We proceed by verifying the conditions of the theorem, beginning with (2). Recall that W = (X, U) and V = (X̃, U) where X̃ = X̃ (X). Consider the classes of functions

$${\psi }_{1;\theta ,\eta ,\alpha }(w,\xi )=\frac{\xi }{{\pi }_{\alpha }(v)}{\stackrel{.}{\ell }}_{\theta ,\eta }(x),$$ (14)

$${\psi }_{2;\theta ,\eta ,\alpha ,h}(w,\xi )=\frac{\xi }{{\pi }_{\alpha }(v)}{B}_{\theta ,\eta }h(x),$$ (15)

for θ ∈ Θ, η ∈ Ξ, and h ∈ ℋ. Then showing (2) is equivalent to showing that

$$\underset{\theta \in \mathrm{\Theta },\eta \in \mathrm{\Xi }}{sup}\mid {\mathbb{G}}_N({\psi }_{1;\theta ,\eta ,\widehat{\alpha }}-{\psi }_{1;\theta ,\eta ,{\alpha }_0})\mid {\to }_{p^{\ast }}0\text{and}$$ (16)

$$\underset{\theta \in \mathrm{\Theta },\eta \in \mathrm{\Xi },h\in \mathcal{H}}{sup}\mid {\mathbb{G}}_N({\psi }_{2;\theta ,\eta ,\widehat{\alpha },h}-{\psi }_{2;\theta ,\eta ,{\widehat{\alpha }}_0,h})\mid {\to }_{p^{\ast }}0.$$ (17)

Under the condition (13) imposed by Breslow & Wellner (2007), (16) holds by virtue of

$$\begin{array}{l}{\psi }_{1;\theta ,\eta ,\widehat{\alpha }}(w,\xi )-{\psi }_{1;\theta ,\eta ,{\alpha }_0}(w,\xi )\\ =\left(\frac{\xi }{{\pi }_{\widehat{\alpha }}(v)}-\frac{\xi }{{\pi }_{{\alpha }_0}(v)}\right){\stackrel{.}{\ell }}_{\theta ,\eta }(x)\\ =\xi \left(\frac{1}{{\pi }_{\widehat{\alpha }}(v)}-\frac{1}{{\pi }_{{\alpha }_0}(v)}-\frac{-{\stackrel{.}{\pi }}_0{(v)}^T}{{\pi }_0^2(v)}(\widehat{\alpha }-{\alpha }_0)\right){\stackrel{.}{\ell }}_{\theta ,\eta }-\xi \frac{{\stackrel{.}{\pi }}_0{(v)}^T}{{\pi }_0^2(v)}(\widehat{\alpha }-{\alpha }_0){\stackrel{.}{\ell }}_{\theta ,\eta }.\end{array}$$

Then

$$\begin{array}{l}\sqrt{N}({\mathbb{P}}_N-P)({\psi }_{1;\theta ,\eta ,\widehat{\alpha }}-{\psi }_{1;\theta ,\eta ,{\alpha }_0})\\ =\sqrt{N}({\mathbb{P}}_N-P)\left(\xi \left(\frac{1}{{\pi }_{\widehat{\alpha }}(v)}-\frac{1}{{\pi }_{{\alpha }_0}(v)}-\frac{-{\stackrel{.}{\pi }}_0{(v)}^T}{{\pi }_0^2(v)}(\widehat{\alpha }-{\alpha }_0)\right){\stackrel{.}{\ell }}_{\theta ,\eta }\right)\\ +\sqrt{N}({\mathbb{P}}_N-P)\left(-\xi \frac{{\stackrel{.}{\pi }}_0{(v)}^T}{{\pi }_0^2(v)}(\widehat{\alpha }-{\alpha }_0){\stackrel{.}{\ell }}_{\theta ,\eta }\right)\\ \equiv R_N+S_N\end{array}$$

where, using (13),

$$\begin{array}{l}\mid R_N\mid \le \sqrt{N}({\mathbb{P}}_N+P)\left\{\mid \left(\frac{1}{{\pi }_{\widehat{\alpha }}(v)}-\frac{1}{{\pi }_{{\alpha }_0}(v)}-\frac{-{\stackrel{.}{\pi }}_0{(v)}^T}{{\pi }_0^2(v)}(\widehat{\alpha }-{\alpha }_0)\right){\stackrel{.}{\ell }}_{\theta ,\eta }\mid \right\}\\ \le {\mathbb{P}}_N(\psi (V)\mid {\stackrel{.}{\ell }}_{\theta ,\eta }\mid )\sqrt{N}\mid \widehat{\alpha }-{\alpha }_0{\mid }^{1+\zeta }\\ +P(\psi (V)\mid {\stackrel{.}{\ell }}_{\theta ,\eta }\mid )\sqrt{N}\mid \widehat{\alpha }-{\alpha }_0{\mid }^{1+\zeta }\\ \le \left\{\sqrt{{\mathbb{P}}_N{\psi }^2(V)·{\mathbb{P}}_N{F}_1^2}+\sqrt{P{\psi }^2(V)·P{F}_1^2}\right\}\sqrt{N}\mid \widehat{\alpha }-{\alpha }_0{\mid }^{1+\zeta }\\ =O_p(1)o_p(1)\end{array}$$

uniformly in θ ∈ Θ, η ∈ Ξ. Here F₁ is a square integrable envelope function for the class of functions {ℓ̇_{θ, η}: θ ∈ Θ, η ∈ Ξ}, which exists by A1^*.

To handle S_N, note that

$$\begin{array}{l}\mid S_N\mid =\mid \sqrt{N}(P_N-P)\left(-\xi \frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0^2}(\widehat{\alpha }-{\alpha }_0){\stackrel{.}{\ell }}_{\theta ,\eta }\right)\mid \\ =\mid (P_N-P)\left(-\xi {\stackrel{.}{\ell }}_{\theta ,\eta }\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0^2}\right)\sqrt{N}(\widehat{\alpha }-{\alpha }_0)\mid \\ \le \underset{\theta \in \mathrm{\Theta },\eta \in \mathrm{\Xi }}{sup}\mid (P_N-P)\left(-\xi {\stackrel{.}{\ell }}_{\theta ,\eta }\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0^2}\right)\mid \sqrt{N}\mid (\widehat{\alpha }-{\alpha }_0)\mid \\ =o_p^{\ast }(1)O_p(1)=o_p^{\ast }(1)\end{array}$$

uniformly in θ ∈ Θ, η ∈ Ξ as the class of functions { $\xi {\stackrel{.}{\ell }}_{\theta ,\eta }{\stackrel{.}{\pi }}_0^T/{\pi }_0^2:\theta \in \mathrm{\Theta }$, η ∈ Ξ} is a Glivenko–Cantelli class of functions. Here is the argument: as the class {ℓ̇_{θ, η}: θ ∈ Θ, η ∈ Ξ} is P-Donsker, it is P-Glivenko–Cantelli. Furthermore the (vector of) function(s) { $-\xi {\stackrel{.}{\pi }}_0^T/{\pi }_0^2$} is square-integrable: for 1 ≤ j ≤ q = dim(α)

$$E{\left(\xi \frac{{\stackrel{.}{\pi }}_{0j}}{{\pi }_0^2}\right)}^2=E\frac{{\stackrel{.}{\pi }}_{0j}^2}{{\pi }_0^3}\le \frac{1}{{\sigma }^2}E\frac{{\stackrel{.}{\pi }}_{0j}^2}{{\pi }_0(1-{\pi }_0)}=\frac{1}{{\sigma }^2}E{\stackrel{.}{\ell }}_{\alpha ,j}^2<\infty$$

by our assumptions on the model π_α(ν). Thus the Glivenko–Cantelli preservation theorem of van der Vaart & Wellner (2000) applies by taking ϕ (u, ν) = uν, ${\mathcal{F}}_1=\{\xi {\stackrel{.}{\pi }}_{0j}/{\pi }_0^2\}$, ℱ₂ = ℓ̇_{θ, η}, and noting that ℱ₁ · ℱ₂ has integrable envelope function ( ${\stackrel{.}{\pi }}_{0j}/{\pi }_0^2$) F₁. A similar argument works for (17) using the square integrable envelope F₂ for {B_{θ, η}h: θ ∈ Θ, h ∈ ℋ, η ∈ Ξ}.

Now note that A1^*, A2, A3^* and A4 imply that ψ_{θ0, η0, α0} ⇝ ψ_{θ0, η0, α0} in ℓ^∞(ℋ) and that (3) holds as α₀ is fixed in both cases. The hypothesized uniform Fréchet differentiability holds under (13) and A3^*: writing θ for (θ, η) in the spirit of theorem 1,

$$\begin{array}{l}\mid \mid \mathrm{\Psi }(\theta ,\alpha )h-\mathrm{\Psi }({\theta }_0,\alpha )h-\stackrel{.}{\mathrm{\Psi }}({\theta }_0,\alpha )(\theta -{\theta }_0)h\mid {\mid }_{\mathcal{H}}\\ =\underset{h\in \mathcal{H}}{sup}\mid P_0\left\{\frac{{\pi }_{{\alpha }_0}}{{\pi }_{\alpha }}\left({\psi }_{\theta ,h}-{\psi }_{{\theta }_0,h}-{\stackrel{.}{\psi }}_{{\theta }_0,h}(\theta -{\theta }_0)\right)\right\}\mid \\ \le {\left\{P_0{\left(\frac{{\pi }_{{\alpha }_0}}{{\pi }_{\alpha }}\right)}^2\right\}}^{1/2}\underset{h\in \mathcal{H}}{sup}{\left\{P_0{\left({\psi }_{\theta ,h}-{\psi }_{{\theta }_0,h}-{\stackrel{.}{\psi }}_{{\theta }_0,h}(\theta -{\theta }_0)\right)}^2\right\}}^{1/2}\\ \le Ko(\mid \mid \theta -{\theta }_0\mid \mid )\end{array}$$

by using the assumed regularity of π_α, (13) and (3) of Breslow & Wellner (2007) to bound P₀(π_α0/π_α)² uniformly in a neighbourhood of α₀ and using A3^* to bound the second term. The additional hypotheses in (ii) and (iv) also follow from the regularity of π_α and (13).

To complete the proof, write ψ _{θ, η, α, h} = (ψ_{1;θ, η, α}, ψ_{2;θ, η, α, h}) as defined in (14) and (15). Then the corresponding components of

$${{\stackrel{.}{\mathrm{\Psi }}}_{\alpha }h=\frac{\partial }{\partial {\alpha }^T}{P}_0{\psi }_{\theta ,\eta ,\alpha ,h}\mid }_{\alpha ={\alpha }_0}$$

are

$$\begin{array}{l}{\stackrel{.}{\mathrm{\Psi }}}_{1,\alpha }=-P_0\left({\stackrel{.}{\ell }}_0{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right)\text{and}\\ {\stackrel{.}{\mathrm{\Psi }}}_{2;\alpha }h=-P_0\left(B_{0}h{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right),h\in \mathcal{H}.\end{array}$$

Consequently, operating with the partitioned (assumption A5, η a measure) version of Ψ̇₀ on both left- and right-hand sides of (7) we find

$$\begin{array}{l}-I_0\sqrt{N}\left({\widehat{\theta }}_N-{\theta }_0\right)-\sqrt{N}({\widehat{\eta }}_N-{\eta }_0)B_0^{\ast }{\stackrel{.}{\ell }}_0\\ =-I_0\sqrt{N}\left({\widehat{\theta }}_N^0-{\theta }_0\right)-\sqrt{N}({\widehat{\eta }}_N^0-{\eta }_0)B_0^{\ast }{\stackrel{.}{\ell }}_0+P_0\left({\stackrel{.}{\ell }}_0{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right)\sqrt{N}({\widehat{\alpha }}_N-{\alpha }_0)+o_p^{\ast }(1)\end{array}$$

and

$$\begin{array}{l}-P_0(B_{0}h){\stackrel{.}{\ell }}_0^T\sqrt{N}\left({\widehat{\theta }}_N-{\theta }_0\right)-\sqrt{N}({\widehat{\eta }}_N-{\eta }_0)B_0^{\ast }{B}_{0}h\\ =-P_0(B_{0}h){\stackrel{.}{\ell }}_0^T\sqrt{N}\left({\widehat{\theta }}_N^0-{\theta }_0\right)-\sqrt{N}({\widehat{\eta }}_N^0-{\eta }_0)B_0^{\ast }{B}_{0}h+P_0\left(B_{0}h{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right)\sqrt{N}({\widehat{\alpha }}_N-{\alpha }_0)\\ +o_p^{\ast }(1).\end{array}$$

Following closely again section 25.12 of van der Vaart (1998) we choose $h={(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }{\stackrel{.}{\ell }}_0$ and subtract the first equation from the second to find

$$\begin{array}{l}{P}_0\left[(I-B_0{(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }){\stackrel{.}{\ell }}_0{\stackrel{.}{\ell }}_0^T\right]\sqrt{N}\left({\widehat{\theta }}_N-{\theta }_0\right)\\ =P_0\left[(I-B_0{(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }){\stackrel{.}{\ell }}_0{\stackrel{.}{\ell }}_0^T\right]\sqrt{N}\left({\widehat{\theta }}_N^0-{\theta }_0\right)\\ -P_0\left[(I-B_0{(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }){\stackrel{.}{\ell }}_0{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right]\sqrt{N}\left({\widehat{\alpha }}_N-{\alpha }_0\right)+o_p^{\ast }(1).\end{array}$$

Recognizing that $I_0=P_0{\stackrel{.}{\ell }}_0{\stackrel{.}{\ell }}_0^T$ is the ordinary information for θ, $(I-B_0{(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }){\stackrel{.}{\ell }}_0$ is the efficient score and ${\stackrel{\sim }{I}}_0=P_0[(I-B_0{(B_0^{\ast }{B}_0)}^{-1}{B}_0^{\ast }){\stackrel{.}{\ell }}_0{\stackrel{.}{\ell }}_0^T]$ is the efficient information, we multiply both sides of the preceding equation by ${\stackrel{\sim }{I}}_0^{-1}$ to find

$$\sqrt{N}\left({\widehat{\theta }}_N(\widehat{\alpha })-{\theta }_0\right)=\sqrt{N}\left({\widehat{\theta }}_N({\alpha }_0)-{\theta }_0\right)-P_0\left({\stackrel{\sim }{\ell }}_0{\mathbf{1}}_{{\mathcal{V}}_0^c}\frac{{\stackrel{.}{\pi }}_0^T}{{\pi }_0}\right)\sqrt{N}({\widehat{\alpha }}_N-{\alpha }_0)+o_p(1)$$ (18)

which is the second hypothesis (1.2) of Pierce (1982), equivalent to our (12) above, with $B=-P_0\left({\stackrel{\sim }{\ell }}_0{\mathbf{1}}_{{\mathcal{V}}_0^c}{\stackrel{.}{\pi }}_0^T/{\pi }_0\right)$. Theorem 2 now follows from theorem 1 via (9) and (18). The resolution of the gap in Breslow & Wellner’s (2007) argument, namely the demonstration that

$$\sqrt{N}({\widehat{\theta }}_N(\widehat{\alpha })-{\theta }_0)=\sqrt{N}{\mathbb{P}}_N^{\widehat{\pi }}{\stackrel{\sim }{\ell }}_0+o_p(1),$$

is obtained from (9), (11) and (18) as a corollary to theorem 2.