On Asymptotically Optimal Tests Under Loss of Identifiability in Semiparametric Models

Abstract

We consider tests of hypotheses when the parameters are not identifiable under the null in semiparametric models, where regularity conditions for profile likelihood theory fail. Exponential average tests based on integrated profile likelihood are constructed and shown to be asymptotically optimal under a weighted average power criterion with respect to a prior on the nonidentifiable aspect of the model. These results extend existing results for parametric models, which involve more restrictive assumptions on the form of the alternative than do our results. Moreover, the proposed tests accommodate models with infinite dimensional nuisance parameters which either may not be identifiable or may not be estimable at the usual parametric rate. Examples include tests of the presence of a change-point in the Cox model with current status data and tests of regression parameters in odds-rate models with right censored data. Optimal tests have not previously been studied for these scenarios. We study the asymptotic distribution of the proposed tests under the null, fixed contiguous alternatives and random contiguous alternatives. We also propose a weighted bootstrap procedure for computing the critical values of the test statistics. The optimal tests perform well in simulation studies, where they may exhibit improved power over alternative tests.

1. Introduction

In this paper we investigate nonstandard testing problems involving a family of probability distributions {P_θ, θ ∈ Θ}, known up to a parameter θ, in a parameter space Θ. The parameter space Θ is assumed to be a subset of an infinite-dimensional metric space. The null and alternative hypotheses are:

$$H_0:\theta \in {\mathrm{\Theta }}_0\text{vs}.H_1:\theta \in \mathrm{\Theta }\backslash {\mathrm{\Theta }}_0,$$

where Θ₀ is a subset of Θ and contains at least two elements. In the usual testing framework, the parameters are unique under the null, so that identifiability is not an issue. While we allow multiple values of θ satisfying the null, we assume that the null distribution, denoted by P₀, is unique, where Θ₀ = {θ ∈ Θ : P_θ = P₀}. Under this set-up, the true value of θ is not identifiable under the null, since for any θ ≠ θ′ in Θ₀, P_θ = P_θ′ = P₀. Such loss of identifiability occurs in diverse applications in the social, biological, physical and medical sciences. We next present two such examples followed by a description of the main contributions of this paper. The Introduction concludes with a brief outline of the remainder of the paper.

1.1. Example 1: univariate frailty regression under right censoring

Let T be a non-negative random variable representing the failure time, C be the independent censoring time, V ≡ min(T, C) and Z ≡ Z(·) be a corresponding p-dimensional covariate process. The observed data {X_i = (V_i, Δ_i, Z_i), i = 1, …, n} consists of n i.i.d. realizations of X = (V, Δ, Z), where Δ ≡ 1{T ≤ V}, 1{·} is the indicator function. In this model, the hazard function of the survival time T given covariates Z is

$$\mathrm{\lambda }\{t;Z(t),W\}=\eta (t)Wexp\{{\beta }^{T}Z(t)\},$$ (1)

where t is the time index, W is an unobserved gamma frailty with mean 1 and variance ζ, β is a p–dimensional regression parameter and η(·) is a completely unspecified baseline hazard function.

When β is not zero, the odds-rate model has been treated extensively; see Kosorok et al. (2004); Murphy et al. (1997); Murphy and van der Vaart (1997, 2000); Parner (1998); Slud and Vonta (2004), among others. Scharfstein et al. (1998) considered semiparametric efficient estimation in the setting where the covariates are time-independent, ζ is assumed known and η(·) is assumed to be absolutely continuous. Bagdonavičius and Nikulin (1999) considered estimation for a class of proportional hazards model, which includes the odds-rate model with ζ unspecified, based on a modified partial likelihood. Kosorok et al. (2004) considered robust inference for odds-rate models when the frailty distribution and regression covariates may be mis-specified. To our knowledge, problems associated with testing the null β = 0 when the frailty parameter is unknown have not been previously considered in the statistical literature.

It has been shown that ζ and η(·) are not identifiable under the null (Kosorok et al., 2004). Intuitively, when β = 0, the covariate process Z provides no information for the failure time process. The frailty W and the baseline hazard η(·) are not distinguishable from each other, hence ζ and η(·) are not identifiable. Thus, the testing problem described above is nonregular and standard asymptotic results are not applicable.

1.2. Example 2: change-point regression for current status data

Change-point models have been studied extensively and have proven to be popular in clinical research. In many settings, a change-point effect is realistic and can be much easier to interpret than a quadratic or more complex nonlinear effect (Chappell, 1989). Change-point Cox models have been widely used in survival applications, as in Kosorok and Song (2007); Luo et al. (1997); Pons (2003), where likelihood ratio tests were investigated. However, to our knowledge, optimal testing has not been explored for such models.

Under current status censoring, a subject is examined once at a random observation time V and at that time it is observed whether the event time T ≤ V or not. The observed data {X_i = (V_i, Δ_i, Z_i), i = 1, …, n} consists of n i.i.d. realizations of X = (V, Δ, Z), where Δ ≡ 1{T ≤ V} and Z is a d–dimensional covariate. Here we let d = 1 for simplicity. In this example, we assume that the time to event T satisfies a change-point Cox model conditionally on the covariate Z. That is, the density of X is given by:

$$p_{\theta }(x)={\left(1-e^{-e^{r_{\gamma }(z)}\mathrm{\Lambda }(v)}\right)}^{\mathrm{\Delta }}{\left(e^{-e^{r_{\gamma }(z)}\mathrm{\Lambda }(v)}\right)}^{1-\mathrm{\Delta }}{f}_{V,Z}(v,z),$$ (2)

with r_γ(z) = αz + (β₁ + β₂z)1{z > ζ}, where α, β₁ and β₂ are scalar regression parameters, ζ is the change-point parameter and Λ(·) is the cumulative baseline hazard function. We also define the collected parameters β ≡ (β₁, β₂), ξ ≡ (β, α), γ ≡ (ξ, ζ) and η ≡ (α, Λ). We are particularly interested in the hypothesis test of the existence of a change-point for regression parameters in the above model, that is, H₀ : β = 0.

Although Cox regression with current status data was discussed by Huang (1996) and others, change-point Cox regression has not been studied with current status data. The development of optimal tests in the current status setting is further complicated by the fact that the nuisance parameter Λ cannot be estimated at the parametric rate, unlike with right censored data.

In model (2), the change-point parameter is present only under the alternative. This is different from Example 1, where the odds rate parameter ζ and the baseline hazard function η(·) are both present, but indistinguishable, under the null.

1.3. Description of main contribution

The statistical literature contains numerous precedents on the nonidentifiability problem in parametric models, see Chernoff (1954), Chernoff and Lander (1995), Dacunha-Castelle and Gassiat (1999) and Liu and Shao (2003). Among others, Dacunha-Castelle and Gassiat (1999) proposed a locally conic parametrization approach to enable asymptotic expansions of the likelihood ratio test under loss of identifiability under the null. Liu and Shao (2003) derived a quadratic approximation of the loglikelihood ratio function by using Hellinger distance. Most authors directly study the approximation of the log-likelihood ratio function in some neighborhood and obtain its asymptotic null distribution. However, the asymptotic optimality properties of the classical likelihood ratio tests (LRT) do not hold anymore (Lindsay, 1995) and Wald and score tests are not even well defined in these nonstandard problems. To our knowledge, all results for testing nonidentifiable P₀ using likelihood based tests are for parametric models. The main aim of this paper is to investigate the construction of optimal likelihood based tests for semiparametric models.

A key question which arises, as noted by Dacunha-Castelle and Gassiat (1999), is: since the parameter is not identifiable, around which point can an expansion be made? To address this question, we assume the existence of a “full rank” reparameterization which contains all the information of the null model and in which all parameters are identifiable. To be specific, we partition θ ≡ (ψ, ζ) and ψ ≡ (β, η), where β ∈ ℝ^p is a parameter of interest, ζ ∈ ℝ^q and η is a parameter defined on an arbitrary parametric space, _η. We assume that the information in the null model can be absorbed into the parameter space of η, through this full rank reparameterization. This is made precise in Section 2. Note that Example 1 requires such a reparameterization since both ζ and η are present under the null. In contrast, such a reparameterization is not required for Example 2 since ζ is not present under the null.

When the models involved are parametric, a special case when η does not depend on ζ under the null, that is, ζ is only present under the alternative, has been studied extensively by Andrews and Ploberger (1994); Davies (1977, 1987); King and Shively (1993), and others. Davies (1977) showed that the likelihood ratio test is optimal in the sense that as the significance level of the test tends to zero, its power function approaches that of the optimum test when ζ is given. These optimality results are very weak and do not provide any guidance regarding the performance of the test in practical applications, where the significance level is fixed, eg. at level .05 (Andrews, 1999). Andrews and Ploberger (1994) studied optimal tests for parametric models using the weighted average power criterion originally introduced by Wald (1943) when studying the likelihood ratio test under regularity conditions, where the model is identifiable under the null. Under loss of identifiability, the likelihood ratio test is generally less powerful than the optimal test in Andrews and Ploberger (1994). These optimal tests possess a Bayesian interpretation, where the weight corresponds to a prior on the nonidentifiable parameter, and are asymptotically equivalent to a Bayesian posterior odds ratio.

In this paper, we adapt the weighted average power criterion (Andrews and Ploberger, 1994; Wald 1943) to construct optimal tests in semiparametric models under loss of identifiability. Our main contribution is to extend the results of Andrews and Ploberger (1994) in at least four directions.

First, Andrews and Ploberger (1994) address only parametric models, as is the case for most of the literature on testing problems with nonidentifiability under the null. Our optimality results are available for semiparametric models, where η may be infinite dimensional and ζ may not be estimable at the usual parametric rate under either the null or the alternative. A semiparametric profile likelihood approach is adopted to reduce the infinite-dimensional model to a finite-dimensional uniformly least-favorable submodel; see Murphy and van der Vaart (2000) for a discussion of profile likelihood in regular settings. We note however, that the idea of uniformly least favorable submodels is a new concept in semiparametric settings, which is not discussed in Murphy and van der Vaart (2000). The development of this concept is both nontrivial and critical to establishing an appropriate optimality criterion for semiparametric models under loss of identifiability.

Second, the results of Andrews and Ploberger (1994) are applicable only for tests where a nuisance parameter (namely ζ) is present only under the alternative. This may not be true in our situation, where a nondegenerate reparameterization may be needed to make ζ vanish under the null. Furthermore, our tests and the optimality results do not depend on the reparameterization.

Third, Andrews and Ploberger (1994) establish that their test is optimal with respect to local alternatives for ψ involving a multivariate normal prior with singular covariance matrix. In our approach, it is only necessary to specify the prior in the direction of β, the parameter of interest, and no prior is needed on the remaining parameter η. This enables us to avoid the singular covariance issue in Andrews and Ploberger (1994).

Fourth, we develop a simple and effective Monte Carlo method of inference for the proposed test statistics.

Adopting a profile likelihood approach has several advantages. First, under the identifiable submodel, the MLE for η may converge at a slower rate than the usual $\sqrt{n}$ rate, such as the change-point Cox model with current status data. This makes the theoretical justification based on Taylor expansion of the full likelihood fail. Second, even if the MLE of the nonparameric component converges at the $\sqrt{n}$ rate, semiparametric likelihoods may not be suitably “differentiable,” in particular, when such a likelihood contains certain empirical terms, as with, for example, the odds-rate model. Third, handling the remainder terms in a Taylor type expansion is challenging, owing to the presence of the infinite dimensional parameters, and a delicate Banach space analysis is required. Employing the profile likelihood enables us to address these issues rigorously.

1.4. Organization of paper

The remainder of the paper is organized as follows. In Section 2, we present the generic testing problem and the model and data assumptions. The optimality results are given in Section 3. We verify that the results hold for the examples in Section 4. In Section 5, we describe a simulation study to evaluate the finite-sample behavior of the proposed tests and to compare its efficiency with some alternative tests for the current status example. In Section 6, we discuss some additional examples without identifiability under the null which are not covered in our current settings and which require further extensions. Proofs are given in Section 7.

2. The hypothesis tests and assumptions

2.1. The optimal tests

In this subsection we formulate the tests of hypotheses when the parameters are not identifiable under the null. Let P_θ denote the probability measure, based on observed data X̃_n ≡ (X₁, X₂, …, X_n), where θ ∈ Θ and the subscript n is the sample size. As mentioned previously, the parameters θ ∈ Θ₀ under the null hypothesis are not identifiable. We assume, as in the examples, that θ can be partitioned as (ψ, ζ), with ζ q-dimensional and ψ of arbitrary dimension. We further assume that ψ can be partitioned as (β, η) so that the null hypothesis can be stated in terms of β, with the nuisance parameter η having arbitrary dimension. The likelihood function of the data is given by l_n(θ) and the profile likelihood for β and ζ is defined as pl_n(β, ζ) = sup_η l_n(β, η, ζ). For the semiparametric model {P_{(β, η, ζ)}} on a sample space , we assume β ∈ ℝ^p, ζ ∈ Ξ, a compact subset of ℝ^q and η ∈ _η, which is a subset of a Banach space.

The hypotheses to be tested are:

$$H_0:\beta ={\beta }_0\text{vs}.H_1:\beta \ne {\beta }_0.$$ (3)

When β = β₀, the null distribution P₀ is unique and the likelihood for a single observation under the null is abbreviated as l⁰. Let π ≡ (η, ζ). The null set of π is Π₀ and its cardinality is the same as that of Ξ, which is at least two. Θ₀ = {β₀} × Π₀. For each ζ ∈ Ξ, η₀(ζ) ≡ {t ∈ _η : (t, ζ) ∈ Π₀} is an interior point of _η. Let ψ₀(ζ) ≡ (β₀, η₀(ζ)), and θ₀(ζ) ≡ (ψ₀(ζ), ζ). Thus, Θ₀ can be represented as Θ₀ = {θ₀(ζ) : ζ ∈ Ξ}.

Before introducing the optimal tests, we need some additional notations for the parameter space and the score and information operators in the semiparametric settings. We denote ${\dot{l}}_{\beta }\in L_2^0(P_{\theta })$ as the derivative of log l₁(θ) with respect to β and l̈_β is the second derivative of log l₁ (θ) with respect to β. $L_2^0(P_{\theta })$ refers to the class of square integrable functions under the measure P_θ with mean 0. The score operator for η is defined as l̇_η, which is a bounded linear map from _η to $L_2^0(P_{\theta })$ with adjoint operator ${\dot{l}}_{\eta }^{\star }:L_2^0(P_{\theta })\mapsto {\overline{\mathcal{H}}}_{\eta }$, where _η is the closed linear span of _η. The information operator is ${\dot{l}}_{\eta }^{\star }{\dot{l}}_{\eta }:L_2^0(P_{\theta })\mapsto L_2^0(P_{\theta })$. The efficient score for β is the ordinary score function l̇_β minus its orthogonal projection onto the closed linear span of the score operator l̇_η. The efficient information for β is ${\stackrel{\sim }{I}}_{\beta }=\int {\stackrel{\sim }{l}}_{\beta }{\stackrel{\sim }{l}}_{\beta }^{\prime }{dP}_{\theta }$, which is the asymptotic variance of the efficient score function.

We use the notations ℙ_n and _n for the empirical distribution and the empirical process of the observations. That is, for every measurable function f and probability measure P,

$${\mathbb{P}}_{n}f=\frac{1}{n}\sum _{i=1}^{n}f(X_i),Pf=\int \mathit{\text{fdP}},{\mathbb{\text{G}}}_{n}f=\frac{1}{\sqrt{n}}\sum _{i=1}^n(f(X_i)-P(f)).$$

We note that although simultaneous estimation of β and ζ fails under the null due to nonidentifiability, estimation results for β̂_n(ζ), the MLE of β at a fixed value of ζ, are often valid under the null. This suggests making inference about β using β̂_n(ζ). For fixed ζ ∈ Ξ, the score, Wald and likelihood ratio test statistics for testing H₀ against H₁ are given by

$$\begin{array}{ccc}\hfill R_n(\zeta )& =& {\mathbb{P}}_n{\dot{l}}_{\beta }({\widehat{\theta }}_0(\zeta ))\prime {\left\{{\mathbb{P}}_n{\dot{l}}_{\beta }{\dot{l}}_{\beta }^{\prime }({\widehat{\theta }}_0(\zeta ))\right\}}^{-1}{\mathbb{P}}_n{\dot{l}}_{\beta }({\widehat{\theta }}_0(\zeta )),\hfill \\ \hfill W_n(\zeta )& =& {({\widehat{\beta }}_n(\zeta )-{\beta }_0)}^{\prime }{\widehat{\stackrel{\sim }{I}}}_{\beta }({\widehat{\theta }}_n(\zeta ))({\widehat{\beta }}_n(\zeta )-{\beta }_0),\text{and}\hfill \\ \hfill L{R}_n(\zeta )& =& -2\left\{l_n({\widehat{\theta }}_0(\zeta ))-l_n({\widehat{\theta }}_n(\zeta ))\right\},\hfill \end{array}$$

where θ̂_n(ζ) ≡ (β̂_n(ζ), η̂_n(ζ), ζ) is the unrestricted MLE of θ at a fixed value of ζ and θ̂₀(ζ) ≡ β₀, η̂₀(ζ), ζ) is the restricted MLE of θ for a fixed value of ζ under the null. ℙ_nl̇_β(θ̂₀(ζ)) = ℙ_nl̇_β(β₀, η̂₀ (ζ), ζ) is the empirical score function of β evaluated at the restricted MLE θ̂₀ (ζ). ℙ_nl̇_β(θ̂_n(ζ)) = ℙ_nl̇_β(β̂_n(ζ), η̂_n(ζ), ζ) is the empirical score function of β evaluated at the unrestricted MLE θ̂_n(ζ). The inverse matrix of ${\widehat{\stackrel{\sim }{I}}}_{\beta }({\widehat{\theta }}_n(\zeta ))$, a consistent estimator of the efficient information Ĩ_β under the null, estimates the covariance matrix of β̂_n(ζ).

The optimal tests we propose take the form:

$$\begin{array}{ccc}\hfill {\text{ER}}_n& =& {(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}{R}_n(\zeta )\right)\mathit{\text{dJ}}(\zeta ),\hfill \\ \hfill {\text{EW}}_n& =& {(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}{W}_n(\zeta )\right)\mathit{\text{dJ}}(\zeta ),\text{and}\hfill \\ \hfill {\text{ELR}}_n& =& {(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}L{R}_n(\zeta )\right)\mathit{\text{dJ}}(\zeta ),\hfill \end{array}$$

where c > 0 is a known constant and J(·) is a pre-selected integrable prior on ζ. Their optimality will be discussed in Section 3. We note that, in semi-parametric settings, the computation of the efficient information may involve high dimensional maximization and nonparametric smoothing. Then the tests ER_n and EW_n may be computationally harder than ELR_n. Hence the likelihood ratio based test ELR_n is more attractive in these settings.

In construction of the optimal tests, understanding and computing θ̂₀(ζ), may be complicated due to the dependence of the parameter θ₀(ζ) on ζ. Assuming the existence of the following full rank reparameterization, we can eliminate the dependence between η and ζ, thereby easing both the theoretical developments and the computations for the proposed tests.

2.2. Full rank reparameterization: breaking the dependence between η and ζ

We assume there exists a map φ_ζ : _η ↦ _η which is one-to-one and uniformly Hadamard-differentiable at η tangentially to _η over ζ ∈ Ξ, i.e.,

$$\underset{(\eta +t_n{h}_n(\zeta ),\zeta )\in {П}_0}{\text{sup}}\Vert \frac{{\phi }_{\zeta }(\eta +t_n{h}_n(\zeta ))-{\phi }_{\zeta }(\eta )}{t_n}-{\dot{\phi }}_{\zeta }(\eta )(h(\zeta ))\Vert \to 0,$$

as sup_ζ∈Ξ ‖h_n(ζ)−h(ζ)‖ → 0, and t_n → 0, where h(ζ) is in the tangent space of _η for all ζ ∈ Ξ and ‖ · ‖ denotes the norm of _η. Its derivative φ̇_ζ is one-to-one and continuously invertible uniformly over ζ ∈ Ξ. That is, there exists a positive constant c such that ‖φ̇_ζ(η₁(ζ) − η₂(ζ))‖ ≥ c‖η₁(ζ) − η₂(ζ)‖ for every η₁(ζ) and η₂(ζ) in _η for all ζ ∈ Ξ. Let η̄ ≡ φ_ζ(η), and ${\ell }_1({\beta }_0,\overline{\eta },\zeta )(x)\equiv l_1({\beta }_0,{\phi }_{\zeta }^{-1}(\overline{\eta }),\zeta )(x)=l^0(x)$, where ζ vanishes under the null, for all x in .

This reparameterization does not change the likelihood, that is, the equality l₁(β, η(ζ), ζ)(x) = ℓ₁(β, η̄, ζ)(x) holds both under the null and the alternative. Under the null, the likelihood l₁(β₀, η₀(ζ), ζ) = ℓ₁ (β₀, η̄₀, ζ) for a specific η̄₀, which does not depend on ζ, and ζ disappears in the null likelihood. We thus reduce the parameter dimension of the null space from Π₀ to _η. For Example 2 in the Introduction, φ can be taken to be the identity and thus the reparameterization is not needed. In contrast, a reparameterization is needed for Example 1. We will give the details later in Section 4.

The reason we assume the existence of such a full rank reparameterization is to eliminate the dependence between η and ζ. The issue is that the optimality results are with respect to a perturbation of the parameter η, which is not well defined in the original space, due to the dependence between parameters η and ζ. Subsequent assumptions are built on the new parameterization θ̄ ≡ (β, η̄, ζ). However, the results still hold for the original parameterization, since the efficient score and efficient information of β are invariant under such reparameterization of η, as given in the following lemma:

Lemma 1. Under the full rank reparameterization, l̃_β(θ) = ℓ̃_β(θ̄), where ℓ̃_β(θ̄) is the efficient score of β under the new reparameterization. The efficient information matrix is also invariant to these reparameterizations.

Remark 1. The full rank reparameterization defined above may not be unique. We will show later in the proof of Theorem 2 that the optimal tests proposed in this paper are invariant to the choice of the full rank reparameterization.

Next we discuss how to construct the optimal tests with the new parameterization, where ζ vanishes, and ψ̄ does not depend on ζ under the null.

2.3. Constructing optimal tests under the full rank reparameterization

Though ζ disappears in the likelihood under the null hypothesis, the score and information are still processes indexed by ζ. For fixed ζ ∈ Ξ, the score, Wald and likelihood ratio test statistics for testing H₀ against H₁ with the new parameterization can be represented as:

$$\begin{array}{ccc}\hfill R_n(\zeta )& =& {\mathbb{P}}_n{\dot{\ell }}_{\beta }{({\widehat{\overline{\psi }}}_0,\zeta )}^{\prime }{\left\{{\mathbb{P}}_n{\dot{l}}_{\beta }{\dot{l}}_{\beta }^{\prime }({\widehat{\overline{\psi }}}_0,\zeta )\right\}}^{-1}{\mathbb{P}}_n{\dot{\ell }}_{\beta }({\widehat{\overline{\psi }}}_0,\zeta ),\hfill \\ \hfill W_n(\zeta )& =& ({\widehat{\beta }}_n-{\beta }_0)\prime {\widehat{\stackrel{\sim }{ℐ}}}_{\beta }({\widehat{\overline{\psi }}}_n,\zeta )({\widehat{\beta }}_n-{\beta }_0),\text{and}\hfill \\ L{R}_n(\zeta )\hfill & =& -2\left\{{\ell }_n({\widehat{\overline{\psi }}}_0,\zeta )-{\ell }_n({\widehat{\overline{\psi }}}_n,\zeta )\right\},\hfill \end{array}$$

where ${\widehat{\overline{\psi }}}_n$ is the unrestricted MLE of ψ̄ and ${\widehat{\overline{\psi }}}_0$ is the restricted MLE of ψ̄₀. ${\mathbb{P}}_n{\dot{\ell }}_{\beta }({\beta }_0,{\widehat{\overline{\eta }}}_0,\zeta )$ is the empirical score function of β evaluated at the restricted MLE ${\widehat{\overline{\psi }}}_0$. ${\mathbb{P}}_n{\dot{\ell }}_{\beta }({\widehat{\beta }}_n,{\widehat{\overline{\eta }}}_n,\zeta )$ is the empirical score function of β evaluated at the unrestricted MLE ${\widehat{\overline{\psi }}}_n$. The inverse matrix of ${\widehat{\stackrel{\sim }{ℐ}}}_{\beta }({\widehat{\overline{\psi }}}_n,\zeta )$, a consistent estimator of the efficient information of β under the null, estimates the covariance matrix of β̂_n. It is thus obvious that the optimal tests are invariant with respect to the choice of full rank reparameterizations.

To further study the asymptotic distribution and the optimality of the proposed tests, we need the following assumptions, based on the full rank reparameterization. We note that except Assumption C, all other assumptions can also be stated with the original parameterization.

2.4. The assumptions based on the reparameterzation

To derive asymptotically optimal tests of H₀, we consider local alternatives to H₀ of the form ${\ell }_n({\beta }_0+h/\sqrt{n},\overline{\eta },\zeta )$ with ζ and η̄ unspecified. The optimality criterion will involve a weighted average power criterion, where the averaging is with respect to an integrable prior Q_ζ(h) on the values of h in ℝ^p defining local alternatives and an integrable prior J(ζ) on ζ. Before formally stating the optimality criterion, we give assumptions on the data and the parameter spaces. The first two assumptions postulate the existence of the prior on local alternatives, Q_ζ(h).

• A1 The efficient information function of β evaluated at (ψ̄₀, ζ), _β(ψ̄₀, ζ), is uniformly continuous in β and ζ over B₀ × Ξ, where B₀ is some neighborhood of β₀. Furthermore, _β(ψ̄₀, ζ) is uniformly positive definite over ζ ∈ Ξ, that is inf_ζ∈Ξ λ_min {_β(ψ̄₀, ζ)} > 0, where λ_min(C) is the smallest eigenvalue of the matrix C.

• A2 Q_ζ is a normal measure with mean β₀ and variance $c{\stackrel{\sim }{ℐ}}_{\beta }^{-1}({\overline{\psi }}_0,\zeta )$ for ζ ∈ Ξ, where c > 0 is a scalar constant.

Assumptions A1 and A2 are analogous to Assumptions 1(e), 1(f) and 4 of Andrews and Ploberger (1994), although there are fundamental differences. Andrews and Ploberger (1994) work directly by building on the full parametric likelihood and their assumptions refer to the information matrix for all parameters. Furthermore, their optimality results are defined in terms of local alternatives for ψ, where the prior is a multivariate normal with singular covariance matrix. Our Assumptions A1 and A2 are only for the parameter of interest, β, with no prior assumptions needed for η under either the null or the alternative.

The next set of conditions assumes the existence of a uniformly least-favorable submodel. This submodel can be viewed as a “uniform” version of the least favorable submodel discussed in Murphy and van der Vaart (2000): the convergence rate of the nuisance parameter now is in the “uniform” sense, and the efficient score and the efficient information possess Donsker and Glivenko-Cantelli properties with “larger” index sets, respectively. When the set of ζ, Ξ, is a singleton, this new submodel concept reduces to the ordinary least favorable submodel. The development of this concept is critical to establishing an appropriate optimality criterion for general semiparametric models under loss of identifiability. Here are the needed assumptions:

• B1 There exists a map t ↦ f_t from a fixed neighborhood of β₀ into _η, such that the map t ↦ ℓ(t, θ̄) defined by ℓ(t, θ̄) ≡ ℓ₁(t, f_t, ζ) is twice continuous differentiable. Let ℓ̇(t, θ̄) and ℓ̈(t, θ̄) denote the derivatives with respect to t. The submodel with parameters (t, f_t, ζ) passes through η̄ at t = β, that is, f_β(β, η̄, ζ) = η̄ for all ζ ∈ Ξ.

• B2 The submodel is uniformly least-favorable at ψ̄₀ = (β₀, η̄₀) and ζ for estimating β₀ in the sense that ℓ̇(β₀, ψ̄₀, ζ) = ℓ̃_β(ψ̄₀, ζ). As (t, β, η̄) → (β₀, β₀, η̄₀), we assume that sup_ζ∈Ξ ‖ℓ̇(t, ψ̄, ζ) − ℓ̃_β(ψ̄₀, ζ)‖ = o_P0 (1) and sup_ζ∈Ξ ‖ℓ̈(t, ψ̄, ζ) − ℓ̈(β₀, ψ̄₀, ζ)‖ = o_P0 (1). In the sequel, we let $o_P^{\mathrm{\Xi }}$ denote a quantity going to zero in probability, under P, uniformly over the set Ξ.

• B3 We assume that ${\widehat{\overline{\psi }}}_0$, the restricted MLE of ψ̄ under the null, satisfies ${\widehat{\overline{\psi }}}_0={\overline{\psi }}_0+o_{P_0}(1)$. The unrestricted MLE ${\widehat{\overline{\psi }}}_n(\zeta )={\overline{\psi }}_0+o_{P_0}^{\mathrm{\Xi }}(1)$. Moreover, let ${\widehat{\overline{\eta }}}_{\beta }(\zeta )\equiv {\text{argmax}}_{\overline{\eta }}{\ell }_n(\beta ,\overline{\eta },\zeta )$, that is $p{\ell }_n(\beta ,\zeta )={\ell }_n(\beta ,{\widehat{\overline{\eta }}}_{\beta }(\zeta ),\zeta )$. Assume that for any random sequences β̃_n → _P0 β₀, we have ${\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta )={\overline{\eta }}_0+o_{P_0}^{\mathrm{\Xi }}(1)$ and the following uniform “no-bias” condition holds:

  $$P_0\dot{\ell }({\beta }_0,{\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )=o_{P_0}^{\mathrm{\Xi }}(\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +n^{-1/2}).$$ (4)

• B4 There exist neighborhoods U of β₀ and V of ψ̄₀, such that the class of functions {ℓ̇(t, ψ̄, ζ) : t ∈ U, ψ̄ ∈ V, ζ ∈ Ξ} is P₀-Donsker with square integrable envelope function and the class of functions {ℓ̈(t, ψ̄, ζ) : t ∈ U, ψ̄ ∈ V, ζ ∈ Ξ} is P₀-Glivenko-Cantelli and is bounded in L₁(P₀), where L₁(P_θ) refers to the class of integrable functions under P_θ.

Assumptions B1-B4 set the stage for the quadratic expansion of the profile likelihood and the derivation of the optimality properties of the proposed tests. Note that these assumptions can also be built on the original parameterization, but we use the new parameterization for ease of presentation. Since our formulation includes parametric models as special cases, the existence of a uniformly least-favorable submodel in our set-up covers all situations considered by Andrews and Ploberger (1994).

Compared with Andrews and Ploberger (1994), we have a stronger form of the unbiasedness condition and stronger requirements on the consistency of the estimators for the expansion of the profile likelihood. This is partly due to the more general structure of the semiparametric model. As in Assumption B3, we require that if β̃_n is any sequence of estimators consistent for β₀, ${\widehat{\overline{\eta }}}_{\zeta }({\stackrel{\sim }{\beta }}_n)$ must be consistent for η̄₀, the true value of the nuisance parameter η̄, uniformly over Ξ. In Andrews and Ploberger (1994), consistency is only needed for the unconstrained MLE (Assumption 2) and the constrained MLE under the null hypothesis (Assumption 3).

To evaluate the local asymptotic distribution of the proposed tests, we require differentiability in quadratic mean (DQM) of the parameters ψ̄, as stated in the following Assumption C, which is commonly used to evaluate the local power. It will be verified for the two examples presented in the introduction. Unlike Assumptions B1–B4, the full rank reparameterization is indispensable in Assumption C:

• C Differentiability in quadratic mean of the parameter ψ̄. A perturbation of ψ̄ in its domain is ψ̄_t = ψ̄₀ + th + o(1), where h ≡ (h_β, h_η̄), h_β ∈ ℝ^p and h_η̄ ∈ _η. The DQM condition for ψ̄₀ with respect to the collection of paths {ψ̄_t} is:

  $${\int \left[\frac{{(d{P}_{{\overline{\psi }}_t,\zeta })}^{1/2}-{(d{P}_0)}^{1/2}}{t}-\frac{1}{2}(A_{\zeta }h)d{P}_0^{1/2}\right]}^2\to 0,\text{as}t\to 0,$$

  for all ζ ∈ Ξ, where A_ζ is a bounded linear operator defined on ℝ^p × _η and takes values in $L_2^0(P_{\theta })$.

Differentiability in quadratic mean implies that the range of A_ζ is contained in $L_2^0(P_{\theta })$. Note that A_ζh = ∂/∂tℓ₁(ψ̄_t, ζ)|_t=0, following similar arguments as in Kosorok and Song (2007), where h = (h_β, h_η̄). We define A_ζ to be given by $A_{\zeta }(h_{\beta },h_{\overline{\eta }})={\dot{\ell }}_{\beta }^{\prime }(\overline{\psi },\zeta )h_{\beta }+{\dot{\ell }}_{\overline{\eta }}(\overline{\psi },\zeta )h_{\overline{\eta }}$, where ℓ̇_β and ℓ̇_η̄ are the score operators for β and η̄ respectively. Moreover, ℝ^p × _η is a Hilbert space with ‖ · ‖ denoting its norm and 〈 ·, ·〉 denoting its inner product. Since in parametric settings, twice continuous differentiability implies DQM (Pollard, 1995), this assumption is weaker than Assumption 1(c) in Andrews and Ploberger (1994).

3. Main results

This section includes several main results. The first one gives the asymptotic null distribution of the proposed tests.

3.1. The distributions of the test statistics under the null

To establish the asymptotic null distribution of the test statistics, a key result about the uniform profile likelihood expansion is summarized in the following lemma.

Lemma 2. Under Assumptions A–C, for any random sequence β̃_n →P₀ β₀,

$$\begin{array}{l}\text{log}p{l}_n({\stackrel{\sim }{\beta }}_n,\zeta )=\text{log}p{l}_n({\beta }_0,\zeta )+n{({\stackrel{\sim }{\beta }}_n-{\beta }_0)}^{\prime }{\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta ))\\ -\frac{1}{2}n{({\stackrel{\sim }{\beta }}_n-{\beta }_0)}^{\prime }{\stackrel{\sim }{I}}_{\beta }({\theta }_0(\zeta ))({\stackrel{\sim }{\beta }}_n-{\beta }_0)+o_{P_0}^{\mathrm{\Xi }}{(\sqrt{n}\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +1)}^2.\end{array}$$ (5)

Lemma 2 enables us to establish the asymptotic equivalence of these test statistics and their asymptotic distributions:

Theorem 1. Under Assumptions A–C, ELR_n = EW_n + o_P0(1) = ER_n + o_P0(1) →_d eχ(c), where

$$e\chi (c)={(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}{\mathbb{G}}^{\prime }({\theta }_0(\zeta )){\stackrel{\sim }{I}}_{\beta }^{-1}({\theta }_0(\zeta ))\mathbb{G}({\theta }_0(\zeta ))\right)\mathit{\text{dJ}}(\zeta ),$$

and (θ₀(ζ)) is the limiting process of _nl̃_β(θ₀ (ζ)), which is a mean zero Gaussian process with variance function σ²(ζ) = Ĩ_β(θ₀(ζ)) indexed by ζ and with covariance function σ²(ζ₁, ζ₂) = P₀ {l̃_β(θ₀(ζ₁)) l̃_β(θ₀(ζ₂))′}, indexed by ζ₁ and ζ₂, for ζ, ζ₁ and ζ₂ ∈ Ξ.

Remark 2. When J(·) does not correspond to a prior on ζ, corresponding rather to a weight function lacking a probabilistic interpretation, then the results in Theorem 1 will generally hold, although the test may no longer possess the optimality discussed in the sequel. Theorem 1 should also hold if Q_ζ(h) is not a prior distribution, corresponding rather to a weight function on local alternatives for β. This robustness indicates that the tests are generally valid under loss of identifiability, yielding a large class of test statistics, with the optimal test being a member of this class.

We note that Theorem 1 only holds for normal weight $Q_{\zeta }^c$, which corresponds to the uniform least favorable direction. As indicated in the proof of Theorem 1, the normal weight function Q_ζ(·) is integrated out, hence does not appear in the test with the original form. Subsequently, the optimal tests depend on the weight function Q_ζ(·) only through the scalar c. The larger c is, the more weight is given to alternatives for which β is large. For example, for a test of the change-point model, larger values of c correspond to greater weight being given to larger changes. In the special case where J(ζ) is a pointmass at a single value ζ₀, the optimal test rejects if and only if LR(ζ₀) exceeds some constant (i.e., the optimal test equals the standard score test for fixed ζ₀) and the optimal test is independent of c. When J(ζ) is not a pointmass distribution, however, the optimal test ELR_n depends on c. The larger c is, the more power is directed at alternatives for which β is large.

The limit as c → 0 of the 2(ELR_n − 1)/c statistic is equal to the “average score” statistic ∫ LR_n(ζ)dJ(ζ), which is the limit of the ELR statistics that are designed for alternatives that are very close to the null hypothesis. At the other extreme, the limit as c → ∞ is log ∫ exp(LR_n(ζ)/2)dJ(ζ). Thus for testing against more distant alternatives, the optimal test statistic is still of an average exponential form.

If the constant c/(1 + c) which appears in the definition of ELR_n is replaced by a constant r > 0, then the limit as r → ∞ of ELR_n is the likelihood ratio test, equivalently, the “sup score” statistic studied in Kosorok and Song (2007). Hence, the sup score test is designed for distant alternatives, but is of a more extreme form than the optimal exponential test, since the latter requires r < 1. It can be easily shown as a corollary to Theorem 1 that the usual likelihood ratio, Wald and score tests have the following distribution:

Corollary 1. Under the null hypotheses and Assumptions A-C, sup_ζ LR_n(ζ) = sup_ζ W_n(ζ) + o_P0 (1) = sup_ζ R_n(ζ) + o_P0 (1) →_d χ, with $\chi ={\text{sup}}_{\zeta }{\mathbb{\text{G}}}^{\prime }({\theta }_0(\zeta )){\stackrel{\sim }{I}}_{\beta }^{-1}({\theta }_0(\zeta ))\mathbb{\text{G}}({\theta }_0(\zeta ))$.

3.2. Optimality of the proposed tests

The second main result of this paper is the optimality property of the proposed tests. Following assumptions in Section 2, we consider local alternatives $\beta ={\beta }_0+h_{\beta }/\sqrt{n}+o(n^{-1/2})$ for h_β ∈ ℝ^p with prior distribution Q_ζ(h_β) on the local alternative direction h_β and prior distribution J(ζ) on the nonidentifiable parameter ζ. The optimality result is as follows:

Theorem 2. Under Assumptions A–C, the test statistics in Theorem 1 are asymptotically uniformly most powerful for testing H₀ : β = β₀ against the contiguous alternative

$$\int d{P}_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^{n}d{Q}_{\zeta }(h_{\beta })dJ(\zeta ),$$

where h ≡ (h_β, h_η(ζ)), $h_{\eta }(\zeta )\equiv {\stackrel{\sim }{q}}_{\zeta }^{\prime }{h}_{\beta }$ and where ${\stackrel{\sim }{q}}_{\zeta }\equiv -{({\dot{\ell }}_{\eta }^{\star }{\dot{\ell }}_{\eta })}^{-}{\dot{\ell }}_{\eta }^{\star }{\dot{\ell }}_{\beta }(\overline{\psi },\zeta )$ is the uniformly least-favorable direction indexed by ζ. Moreover, this optimality result is invariant under the choice of reparameterization.

Theorem 2 also implies that the proposed tests have the greatest weighted average power asymptotically in the class of all tests of asymptotic significance level α, against the alternative $P_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^n$. That is, they maximize

$$\overline{\underset{n\to \infty }{\text{lim}}}\int P({\phi }_n\text{rejects}|{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta )d{Q}_{\zeta }(h_{\beta })dJ(\zeta )$$

over all tests φ_n of asymptotic level α.

Our optimality results are under alternatives ${\beta }_0+h_{\beta }/\sqrt{n}+o(n^{-1/2})$, with nonsingular normal weights on h_β. Our weights on h_β are precisely Andrews and Ploberger's [2] weights projected onto the parameter space that is of interest. Thus our results and Andrews and Ploberger's are consistent.

We now discuss the choice of the direction q_ζ, the priors Q_ζ(·) and J(·). By the Neyman-Pearson lemma, for any appropriate prior distributions Q_ζ(·) and J(·) and any known directions q_ζ, a UMP test for testing H₀ : β = β₀ against the contiguous alternative $\int d{P}_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^{n}d{Q}_{\zeta }(h_{\beta })\mathit{\text{dJ}}(\zeta )$, where h ≡ (h_β, h_η(ζ)), $h_{\eta }(\zeta )=q_{\zeta }^{\prime }{h}_{\beta }$ is defined by

$${\gamma }_n=\{\begin{array}{c}1,\text{if}QL{R}_n>k_{\alpha n},\\ {\mathrm{\lambda }}_n,\text{if}QL{R}_n=k_{\alpha n},\\ 0,\text{if}QL{R}_n<k_{\alpha n,}\end{array}$$

where k_αn > 0, λ_n ∈ [0, 1] are constants such that the rejection probability is α under the null and

$${\text{QLR}}_n=\frac{\int l_n\left({\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta \right)d{Q}_{\zeta }(h_{\beta })\mathit{\text{dJ}}(\zeta )}{l_n^0}.$$

We have the following result:

Corollary 2. Under Assumptions A–C, the null hypothesis and the contiguous alternatives,

$$\mathit{\text{QL}}{R}_n={(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}L{R}_n(\zeta )\right)W(q_{\zeta },\zeta )\mathit{\text{dJ}}(\zeta )+o_p(1),$$

where W(q_ζ, ζ) ≤ 1 is defined in equation (17) in section 7 below. When q_ζ = q̃_ζ, W(q̃_ζ, ζ) = 1 and QLR_n = ELR_n + o_P0 (1).

As the alternatives we consider are contiguous to the null, in each direction q_ζ, which indexes QLR_n, there exists a consistent estimator η̃_n(q_ζ) of η̄₀ by the convolution theorem, provided certain conditions hold. The optimal tests can thus be built on η̃_n(q_ζ).

In applications with composite hypotheses where q_ζ is unknown, there may not exist a direction which can maximize the power over all directions (Bickel et al., 2006). In a regular testing problem where all parameters are identifiable, it can be shown that the likelihood ratio test, which is built on the uniformly least-favorable direction, will maximize the minimum power of all directions of the alternatives, over all the test based directions. In our nonregular testing problem, the situation is further complicated, since the power depends on the covariance structure of (θ₀(ζ)). It is not clear if the maximin property still holds in our problem. We note that, however, our tests can be interpreted as the “maximum direction” test. Moreover, since the power of the test is not affected by multiplying by a constant in QLR_n, we can standardize W(q_ζ, ζ)dJ(ζ) to obtain dJ̃(ζ), which is a probability measure on ζ. Then the question of the optimal choice of both q_ζ and J(ζ) reduces to the question of the optimal choice of J̃(ζ). Hence, without loss of generality we can replace q_ζ with q̃_ζ. For this reason, we should choose q_ζ = q̃_ζ and focus on the choice of Q_ζ(·) and J̃(·) for optimization.

One reason we use the normal weight for Q_ζ in this paper is to facilitate a comparison with Andrews and Ploberger (1994). Using the normal prior with covariance matrix proportional to the efficient information matrix also leads to a significant simplification of the representation of the test statistics, since many terms cancel in the proof of Theorem 1. However we note that the choice of Q_ζ(·) is not limited to the normal weight studied in this paper, as indicated in the proof of Theorem 2. More general choices of the priors Q_ζ(·) and J(·) merit future consideration, but this is beyond the scope of the current paper.

The optimality of the likelihood ratio statistics with loss of identifiability under the null for semiparametric models is of potential interest. Similar to the likelihood ratio test under loss of identifiability with parametric models (Andrews and Ploberger, 1994), in the semiparametric setting, the profile likelihood ratio statistic is not of the optimal average exponential form. It can be shown to be a limit of an average exponential test, but only if a parameter is pushed beyond an admissible boundary, as noted by Andrews and Ploberger (1995) in the parametric case.

3.3. The distributions of the test statistics under local alternatives

To gain insight into the power of the optimal tests in practice, it is worthwhile to study their asymptotic distributions under local alternatives. In the following two theorems, Theorem 3 gives the asymptotic distribution for fixed local alternatives $P_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),{\zeta }_1}^n$, while Theorem 4 gives the asymptotic distribution for random local alternatives $\int d{P}_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^{n}d{Q}_{\zeta }(h_{\beta })\mathit{\text{dJ}}(\zeta )$. As shown in the theorems, the distributions depend on the form of the alternative, which will depend in part on the specifics of the application. These results also usually depend on the prior distributions J(·) and Q_ζ(·), for both fixed alternatives and random alternatives, although in different manners.

Theorem 3. Under local alternatives $P_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),{\zeta }_1}^n$ and Assumptions A–C, ELR_n = EW_n + o_p(1) = ER_n + o_p(1) →_d fχ(c), with

$$\begin{array}{c}f\chi (c)={(1+c)}^{-\frac{p}{2}}\int exp[\frac{1}{2}\frac{c}{1+c}\left\{\mathbb{\text{G}}({\theta }_0(\zeta ))+v_{\star }(h_{\beta },\zeta ,{\zeta }_1)\right\}\prime \\ {\stackrel{\sim }{I}}_{\beta }^{-1}({\theta }_0(\zeta ))\left\{\mathbb{\text{G}}({\theta }_0(\zeta ))+v_{\star }(h_{\beta },\zeta ,{\zeta }_1)\right\}]\mathit{\text{dJ}}(\zeta ),\end{array}$$

where v_⋆(h_β, ζ, ζ₁) ≡ P₀l̃_β(θ₀(ζ))l̃_β(θ₀(ζ₁))′h_β.

Now we establish the asymptotic distribution of the test statistics under the alternative $\int d{P}_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^{n}d{Q}_{\zeta }(h_{\beta })\mathit{\text{dJ}}(\zeta )$:

Theorem 4. Under Assumptions A–C and the local alternative $\int d{P}_{{\overline{\psi }}_0+h/\sqrt{n}+o(n^{-1/2}),\zeta }^{n}d{Q}_{\zeta }(h_{\beta })dJ(\zeta ),\mathit{\text{EL}}{R}_n=E{W}_n+o_p(1)=E{R}_n+o_p(1){\to }_{d}r\chi (c)$, where rχ(c) is a real random variable such that its cumulative distribution function Pr(rχ(c) ≤ t) = P₀ [1{eχ(c) ≤ t}eχ(c)].

3.4. Monte Carlo computation and inference

Although we have obtained the asymptotic distributions of the test statistics, these distributions generally have complicated analytic forms which depend on the values of unknown nuisance parameters. We now introduce a weighted bootstrap method to obtain the asymptotically valid critical values of eχ(c). This method does not require explicit evaluation of the limiting distribution, thereby avoiding the numerical difficulties inherent in such an evaluation.

We first generate n i.i.d. positive random variables κ₁, …, κ_n, with mean 0 < μ_κ < ∞, variance $0<{\sigma }_{\kappa }^2<\infty$ and with ${\int }_0^{\infty }\sqrt{P({\kappa }_1>u)}\mathit{\text{du}}<\infty$. Next, we divide each weight by the sample average of the weights κ̄, to obtain “standardized weights” ${\kappa }_1^{\circ },\dots ,{\kappa }_n^{\circ }$ which sum to n. For a real, measurable function f, define the weighted empirical measure ${\mathbb{\text{P}}}_n^{\circ }f\equiv n^{-1}{\sum }_{i=1}^n{\kappa }_i^{\circ }f(X_i)$. Let ${\widehat{\psi }}_n^{\circ }(\zeta )=({\widehat{\beta }}_n^{\circ }(\zeta ),{\widehat{\eta }}_n^{\circ }(\zeta ))$ denote the maximizer of $l_n^{\circ }(\psi ,\zeta )$ over ψ ∈ Ψ at fixed ζ ∈ Ξ, where $l_n^{\circ }$ is obtained by replacing ℙ_n with ${\mathbb{\text{P}}}_n^{\circ }$ in the definition of l_n. Similarly, let ${\widehat{\psi }}_0^{\circ }(\zeta )=({\widehat{\beta }}_0^{\circ }(\zeta ),{\widehat{\eta }}_0^{\circ }(\zeta ))$ denote the maximizer of ${(l_n^0)}^{\circ }(\psi ,\zeta )$ over ψ ∈ Ψ at fixed ζ ∈ Ξ, where ${(l_n^0)}^{\circ }$ is obtained by replacing ℙ_n with ${\mathbb{\text{P}}}_n^{\circ }$ in the definition of $l_n^0$, the log likelihood under the null. Now repeat the bootstrap procedure a large number of times M̃_n and compute the differences of the bootstrapped unrestricted MLE and restricted MLE of $\beta :d{\widehat{\beta }}_k^{\circ }(\zeta )={\widehat{\beta }}_{n,k}^{\circ }(\zeta )-{\widehat{\beta }}_{0,k}^{\circ }(\zeta ),k=1,\dots ,M_n$, as processes of ζ. Note that we are allowing the number of bootstraps to depend on n. Define $\zeta \mapsto {\widehat{\mu }}_n(\zeta )\equiv {\stackrel{\sim }{M}}_n^{-1}{\sum }_{k=1}^{{\stackrel{\sim }{M}}_n}d{\widehat{\beta }}_k^{\circ }(\zeta )$ and let

$$\zeta \mapsto {\widehat{V}}_n(\zeta )={\stackrel{\sim }{M}}_n^{-1}\sum _{k=1}^{{\stackrel{\sim }{M}}_n}\left(d{\widehat{\beta }}_{1,k}^{\circ }(\zeta )-{\widehat{\mu }}_n(\zeta )\right)\left(d{\widehat{\beta }}_{1,k}^{\circ }(\zeta )-{\widehat{\mu }}_n(\zeta )\right)\prime .$$

To estimate critical values, we compute the standardized bootstrap test statistics $T_{n,k}^{\circ }\equiv {(1+c)}^{-\frac{p}{2}}\times \int exp\left[\frac{1}{2}\frac{c}{1+c}\left\{\left(d{\widehat{\beta }}_{1,k}^{\circ }(\zeta )-{\widehat{\mu }}_n(\zeta )\right)\prime {\widehat{V}}_n^{-1}(\zeta )\left(d{\widehat{\beta }}_{1,k}^{\circ }(\zeta )-{\widehat{\mu }}_n(\zeta )\right)\right\}\right]\mathit{\text{dJ}}(\zeta )$, for 1 ≤ k ≤ M̃_n. For a test of size α, we compare the observed test statistics with the (1 − α)th quantile of the corresponding M̃_n standardized bootstrap statistics. The reason we subtract off the mean is to ensure that we obtain a valid approximation to the null distribution when the null hypothesis may not be true. If not, then there may be loss of power, although the type I error rate will still be controlled when the null is true. The proof of the bootstrap validity can be built upon the proof of Theorems 7−8 in Kosorok and Song (2007). We omit the details.

4. Examples

In this section, we study the two examples in the introduction to illustrate the two types of nonidentifiability settings, one where a nuisance parameter is present under the null and one where it is not. These examples demonstrate important differences in how the full rank reparameterizations and uniformly least favorable submodels are defined in the two settings. We present Example 2 first because a reparameterization is not required, simplifying the presentation.

4.1. Example 2 revisited: change-point regression for current status data

In the change-point Cox model with current status data, a test of the existence of a threshold effect corresponds to a test of the null H₀ : β = 0. The change-point parameter ζ is present only under the alternative. Hence it suffices to take φ_ζ as the identity map.

We make the following assumptions and will argue that the assumptions in Section 2 can be checked under these assumptions. Given Z, T and V are independent, Z belongs to a compact subset of ℝ. The change-point parameter ζ ∈ [a, b], for some known −∞ < a < b < ∞ with Pr(Z < a) > 0 and Pr(Z > b) > 0. Assume P(Var(Z|V)) > 0, which guarantees that, as we will show later, the efficient information (Ĩ)_β(θ₀(ζ)) is positive definite uniformly over ζ ∈ Ξ. The Lebesgue density of V is positive and continuous on its support [σ, τ] with 0 < σ < τ < ∞. The baseline hazard function Λ is continuously differentiable at [σ, τ], with derivative that is bounded away from 0 and satisfies Λ₀(σ) > 0, Λ₀(τ) < M, for some known M. We let _Λ denote a set of non-decreasing cadlag functions Λ on [σ, τ] with Λ(τ) ≤ M.

The likelihood function equals (2) with f_V,Z(v, z) removed, because it can be absorbed into the underlying measure on the sample space. The log-likelihood for a single observation log l₁(θ) takes the form log l₁(θ) = δ log[1 − exp{−Λ(v) exp(r_γ(z))}] − (1 − δ) exp(r_γ(z))Λ(v). Define Z(ζ) ≡ (1{Z > ζ}, Z1{Z > ζ}, Z) and note that with such a data representation we can adopt much material in the literature and hence simplify our arguments.

To define a uniformly least-favorable submodel in β, we take two steps. For Step 1, we calculate scores for ξ and Λ. The score function for ξ is l̇_ξ(x) = z(ζ)Λ(v)Q(x; θ) with

$$Q(x;\theta )=e^{r_{\gamma }(z)}\left[\delta \frac{e^{-e^{r_{\gamma }(z)}\mathrm{\Lambda }(v)}}{1-e^{-e^{r_{\gamma }(z)}\mathrm{\Lambda }(v)}}-(1-\delta )\right].$$

The score operator for Λ along Λ_t = Λ+th with t ≥ 0 and h a non-decreasing non-negative right continuous function, is given by

$${\dot{l}}_{\mathrm{\Lambda }}(h)(x)={\frac{\partial }{\partial t}logp(x;\gamma ,{\mathrm{\Lambda }}_t)|}_{t=0}=h(v)Q(x;\theta ).$$

We project l̇_ξ(X) onto the space generated by l̇_Λ. That is, we need to find a function $h_{\zeta }^{\star }(V)\in H_{\mathrm{\Lambda }}$ such that ${\dot{l}}_{\xi }-{\dot{l}}_{\mathrm{\Lambda }}(h_{\zeta }^{\star })\perp {\dot{l}}_{\mathrm{\Lambda }}(h)$, for all h ∈ _Λ, which is equivalent to solving the least square problem P_θ‖l̇_ξ − l̇_Λh‖². The solution under the null is $h_{\zeta }^{\star }(V)\equiv {\mathrm{\Lambda }}_0(V)h_{\zeta }^{\star \star }(V)$, where $h_{\zeta }^{\star \star }=P(Z(\zeta )Q^2(X;\psi ))/P(Q^2(X;\theta ))$, which is assumed to possess a version that is differentiable componentwise with the derivatives being bounded on [σ, τ] uniformly over ζ ∈ Ξ. It can be shown that Λ_t(θ) is indeed a hazard function when t is sufficiently close to ξ.

The uniformly least-favorable direction for ξ is ${\mathrm{\Lambda }}_t(\theta )=\mathrm{\Lambda }+(\xi -t)\prime \varphi (\mathrm{\Lambda })h_{\zeta }^{\star \star }\circ {\mathrm{\Lambda }}_0^{-1}\circ \mathrm{\Lambda }$. Here ϕ is a function mapping [0, M] into [0, ∞) such that ϕ(y) = y on [Λ₀(σ), Λ₀(τ)] and the function y ↦ ϕ(y)/y is Lipschitz and ϕ(y) ≤ c(min(y, M − y)) for a sufficiently large constant c. The efficient score for ξ for this uniformly least-favorable submodel is given by:

$${\stackrel{\sim }{l}}_{\xi }(x;t,\theta )=\left[z-\frac{\varphi (\mathrm{\Lambda })(v)}{{\mathrm{\Lambda }}_t(\theta )(v)}{h}_{\zeta }^{\star \star }\circ {\mathrm{\Lambda }}_0^{-1}\circ \mathrm{\Lambda }(v)\right]{\mathrm{\Lambda }}_t(\theta )(v)Q(x;t,{\mathrm{\Lambda }}_t(\theta )).$$

${\mathrm{\Lambda }}_0^{-1}$ may be extended to [0, ∞) by setting ${\mathrm{\Lambda }}_0^{-1}(u)=\sigma$ for u ≤ Λ₀(σ) and ${\mathrm{\Lambda }}_0^{-1}(u)=\tau$ for u > Λ₀(τ).

For Step 2, we next project l̇β(x) onto the space generated by l̃_ξ. The efficient score function for β, l̃_β, is the first two coordinates of l̃_ξ minus its projection on the remaining coordinates of l̃_ξ. Since l̃_ξ lies in a finite-dimensional space, the projection path has a matrix representation. The efficient information for ξ, Ĩ_ξ can be partitioned as a two-by-two block matrix, with ${\stackrel{\sim }{I}}_{\xi }^{11}(\theta )$ denoting its first two-by-two principle submatrix, and so on. We define $v_{\theta }^{\prime }=(1,-{({\stackrel{\sim }{I}}_{\xi }^{22})}^{-1}{\stackrel{\sim }{I}}_{\xi }^{21})$, and ξ_t(θ) = ξ − (β − t)v_θ. We also define ${\mathrm{\Lambda }}_t(\theta )=\mathrm{\Lambda }+({\xi }_t(\theta )-t)\prime \varphi (\mathrm{\Lambda })h_{\zeta }^{\star \star }\circ {\mathrm{\Lambda }}_0^{-1}\circ \mathrm{\Lambda }$.

Now we use the uniformly least-favorable path t ↦ (ξ_t(θ), Λ_t(θ)) in the parameter space for the nuisance parameter η ≡ (α, Λ). This leads to l(t, β, α, Λ) = log l(ξ_t(θ), Λ_t(θ)). This submodel is least favorable at (ξ₀, Λ₀) uniformly over ζ ∈ Ξ since ${\partial /\partial t|}_{t={\beta }_0}l(t,{\beta }_0,\alpha ,\mathrm{\Lambda })=v_{\theta }^{\prime }{\stackrel{\sim }{l}}_{\xi }$, Whereas $v_{\theta }^{\prime }{\stackrel{\sim }{l}}_{\xi }={\stackrel{\sim }{l}}_{\beta }$. The efficient information matrix for β is, ${\stackrel{\sim }{I}}_{\beta }={\stackrel{\sim }{I}}_{\xi }^{11}-{\stackrel{\sim }{I}}_{\xi }^{11}{\left({\stackrel{\sim }{I}}_{\xi }^{22}\right)}^{-1}{\stackrel{\sim }{I}}_{\xi }^{21}(\theta )$. The remainder of Assumption B4 can be verified by standard empirical process arguments.

To verify Assumption A1 in Section 2, it suffices to show that Ĩ_ξ is uniformly positive definite over ζ ∈ Ξ, which can be achieved by checking that inf_ζ∈Ξ λ_min{P₀(Cov(Z(ζ)|V))} > 0. We first show that the random vector (Z, 1{Z > ζ}, Z1{Z > ζ}) is linearly independent given V pointwisely in ζ ∈ Ξ. Suppose that given V,

$$aZ+b1\{Z>\zeta \}+cZ1\{Z>\zeta \}=0,$$ (6)

a.s., for some constants a, b and c. Our aim is to show a = b = c = 0. When Z ≤ ζ, (6) becomes aZ1{Z ≤ ζ} = 0. Since Var(Z|V) > 0 and P(Z ≤ ζ|V) > 0, for every ζ ∈ Ξ, Var(Z|Z ≤ ζ, V) > 0, and therefore a = 0. When Z > ζ, (6) becomes (b + cZ)1{Z > ζ} = 0. If c ≠ 0, Z = −b/c, which is contradicted with the fact that Var(Z|Z > ζ, V) > 0. Thus we conclude that c = 0 and b = 0 as a consequence. That P(Cov(Z(ζ)|V)) is uniformly positive definite over ζ ∈ Ξ follows since P(Cov(Z(ζ)|V)) is a continuous function of ζ and Ξ is compact.

The profile likelihood estimator ψ̂_n(ζ) can be shown to be consistent for (β₀, Λ₀) by a similar proof as used for the full maximum likelihood estimator in Huang (1996). The following lemma shows the uniform consistency of ψ̂_n(ζ) under the null.

Lemma 3. ${\widehat{\psi }}_n\left(\zeta \right)-{\psi }_0=o_{P_0}^{\mathrm{\Xi }}(1)$.

To verify the uniform no-bias condition (4), we need the following result about the uniform rate of convergence:

Lemma 4. Suppose that d(η, η₁) : η, η₁ ∈ _η is the metric defined on _η, and C₁, C₂ and C₃ are positive constants with,

$$P_0(m_{\beta ,\eta ,\zeta }-m_{\beta ,{\eta }_0,\zeta })\le -C_1{d}^2(\eta ,{\eta }_0)+C_2{\Vert \beta -{\beta }_0\Vert }^2,\mathit{\text{and}}$$ (7)

$$P_0^{\star }\underset{\beta \in B,\eta \in {\mathcal{H}}_{\eta },\Vert \beta -{\beta }_0\Vert <\delta ,d(\eta ,{\eta }_0)<\delta ,\zeta \in \mathrm{\Xi }}{sup}|{\mathbb{G}}_n(m_{\beta ,\eta ,\zeta }-m_{\beta ,{\eta }_0,\zeta })|\le C_3{\phi }_n(\delta ),$$ (8)

for functions φ_n such that δ ↦ φ_n(δ)/δ^α is decreasing for some α < 2 and sets B×_η×Ξ such that under the null Pr(β̃_n ∈ B, η̂_β̃n (ζ) ∈ _η, ζ ∈ Ξ) → 1. Then ${sup}_{\zeta \in \mathrm{\Xi }}{r}_{n}d({\widehat{\eta }}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),{\eta }_0)\le O_{P_0}^{\star }(1+r_n\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert )$ for any sequence of positive numbers r_n such that $r_n^2{\phi }_n\left(1/r_n\right)\le \sqrt{n}$ for every n.

We apply Lemma 4 with η = (α, Λ), _η = ℝ × _Λ, where _Λ is the closed linear span of _Λ, d(η, η₁) = ‖α − α₁‖ + ‖Λ − Λ₁‖₂ and

$$m_{\beta ,\eta ,\zeta }=\{\begin{array}{l}log\frac{p_{\beta ,\eta ,\zeta }}{p_{{\beta }_0,{\eta }_0}},\text{if}\eta ={\eta }_0,\hfill \\ 2log\frac{p_{\beta ,\eta ,\zeta }+p_{{\beta }_0,{\eta }_0}}{2p_{{\beta }_0,{\eta }_0}},\text{otherwise}.\hfill \end{array}$$

Condition (7) can be established by the Taylor expansion and the uniform boundedness on the derivatives of the loglikelihood. Condition (8) can be verified using Lemma 3.3 of Murphy and van der Vaart (1999), with the choice ${\phi }_n\left(\delta \right)={\delta }^{1/2}\left(1+M{\delta }^{-3/2}/\sqrt{n}\right)$, where M ≥ ‖m_β,η,ζ‖_∞ is a constant. These conditions imply that $\Vert {\widehat{\alpha }}_{{\stackrel{\sim }{\beta }}_n}(\zeta )-{\alpha }_0\Vert +\Vert {\widehat{\mathrm{\Lambda }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta )-{\mathrm{\Lambda }}_0{\Vert }_2=O_p^{\mathrm{\Xi }}(\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +n^{-1/3})$, for any sequence β̃_n → 0. Now we only need to verify

$$P_0\dot{\ell }({\beta }_0,{\beta }_0,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),(\zeta )=o_{P_0}^{\mathrm{\Xi }}(\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +n^{-1/2}),$$ (9)

which is equivalent to (4) under regularity conditions. We further decompose (9) as (17) in Murphy and van der Vaart (2000), which can be easily verified by the Taylor expansion and the uniform boundedness on the first and second derivatives of the loglikelihood.

It is not difficult to see that {p_ξ,Λ(ζ)} is differentiable in quadratic mean at (ψ₀, ζ) with respect to the set of directions {ξ₀ + th₁, Λ₀ + th₂}, where h₁ ∈ ℝ³, and h₂ is a non-decreasing non-negative right continuous function. Thus all conditions in Section 2 are satisfied.

4.2. Example 1 revisited: univariate frailty regression under right censoring

The odds-rate model we consider in this paper posits that the hazard function has the form (1). We define g_ζ(s) ≡ (1 + ζs)^−1/ζ, for ζ > 0, and g₀(s) ≡ lim_ζ↓0 g_ζ(s) = exp(−s). Let S_Z(·) denote the survival function of T given Z, and after integrating over W, S_Z(t) becomes $g_{\zeta }\left({\int }_0^t{e}^{\beta \prime Z(u)}d\eta (u)\right)$, where the cumulative baseline hazard function η(·) is a non-negative, monotone increasing cadlag (right-continuous with left-hand limits) function. We will argue later that Assumptions A–C can be checked under the following conditions. The true null survival function is unique and denoted as S₀. The censoring time C is independent of T given Z and uninformative of ζ and β. Moreover, for a finite time point τ, P₀1{C ≥ τ} = P₀1{C = τ} > 0 almost surely. ζ ∈ Ξ ≡ [0, K₀] for some known K₀ < ∞. The null value β₀ = 0 is an interior point of a known compact set B₀ ∈ ℝ^p. The parameter space for η, _η, is a Banach space consisting of continuous and monotone increasing functions on the interval [0, τ] equipped with the total variation norm ‖ · ‖_v. Its closed linear span is denoted as _η. The function η(·) ∈ _η satisfies η(0) = 0 and η(τ) < ∞. The covariate process Z(·) is uniformly bounded in total variation on [0, τ] and var[Z(0+)] is positive definite.

The true values of π ≡ (η, ζ) are not unique under the null, since the null set Π₀ contains all pairs of (η, ζ) satisfying, for t ∈ [0, τ], (1 + ζη(t))^−1/ζ = S₀(t), when ζ ∈ (0, K₀]; and exp(−η(t)) = S₀(t), when ζ = 0. In this example, ζ appears both under the null and the alternative. Equivalently, for any fixed ζ ∈ (0, K₀], η₀(t)(ζ) = (S₀(t)^−ζ − 1)/ζ and for ζ = 0, η₀(t)(ζ) = − log(S₀(t)), t ∈ [0, τ]. Hence Π₀ = {(ζ, η₀(ζ)) : ζ ∈ Ξ}. Thus we need a suitable parameter transformation for this example. Let η̄ = φ_ζ(η) ≡ (1 + ηζ)^1/ζ − 1, for ζ > 0; and η̄ = lim_ζ→0 φ_ζ(η) = exp(η) − 1. It can be easily checked that η̄ ∈ _η. The following arguments reveal that the map φ_ζ(η) : _η ↦ _η is a full-rank reparameterization.

The log likelihood function with the new parameter θ̄ = (β, η̄, ζ) is

$$\begin{array}{ll}{\ell }_n\left(\overline{\theta }\right)=\hfill & {\mathbb{P}}_n[\delta \{log{\alpha }_1\left(v\right)+\left(\zeta -1\right)log\left(\overline{\eta }\left(v\right)+1\right)\}+\beta \prime z\left(v\right)\hfill \\ \hfill & +\left(1+\delta \zeta \right)log{g}_{\zeta }\left\{{\int }_0^v{e}^{\beta \prime z(s)}{\left(\overline{\eta }\left(s\right)+1\right)}^{\zeta -1}d\overline{\eta }\left(s\right)\right\}],\hfill \end{array}$$ (10)

where a₁(·) is the derivative of η̄(·). We will replace a₁(·) with nΔη̄(·) in the sequel, since this form of the empirical log-likelihood function is asymptotically equal to the true log-likelihood function. When β = 0, it is clear that ζ vanishes since (10)=ℙ_n{δ log Δη̄(v)−(δ+1) log(1+η̄(v))}, and η̄(0) = 0. The odds-rate model with new parameterization ψ̄ ≡ (β, η̄) is identifiable under the null, since the null survival function S₀(t|z) = (1 + η̄)⁻¹ is a strictly monotone function of η̄ and is unique.

The Gâteaux derivative of φ_ζ(η) at η ∈ _η exists and is obtained by differentiating φ_ζ(η) along the submodels t ↦ η + th. This derivative is φ̇_ζ(η)(h) ≡ ∂/∂tφ_ζ(η + th)|_t=0 = (1 + ζη)^1/ζ−1h for ζ > 0 and exp(η)h for ζ = 0.

The Gâteaux differentiability of φ_ζ(η) pointwisely in ζ can be strengthened to uniform Fréchet differentiability by noticing that

$$\underset{t\downarrow 0}{lim}\underset{\zeta \in \mathrm{\Xi }}{sup}\underset{{\Vert h\Vert }_v\le r,h\in {\overline{\mathscr{H}}}_{\eta }}{sup}\left|{\int }_0^1\left\{{\dot{\phi }}_{\zeta }\left(\eta +\mathit{\text{sth}}\left(\zeta \right)\right)-{\dot{\phi }}_{\zeta }\left(\eta \right)\right\}ds\right|=0,$$

for any r > 0. Thus

$${sup}_{\zeta \in \mathrm{\Xi }}{sup}_{\Vert h{\Vert }_v\le r,h\in {\overline{\mathscr{H}}}_{\eta }}{\Vert {\phi }_{\zeta }(\eta +h(\zeta ))-{\phi }_{\zeta }(\eta )-{\dot{\phi }}_{\zeta }(\eta )(h(\zeta ))\Vert }_v/{\Vert h(\zeta )\Vert }_v=o(1),$$

as ‖h(ζ)‖_v → 0 uniformly over ζ ∈ Ξ, which we will hereafter refer to as “uniformly Fréchet differentiable.” Since φ̇_ζ(η)(h) is uniformly bounded and Lipschitz in h, by checking the definition, we can show that φ̇_ζ is one-to-one and continuously invertible uniformly over ζ ∈ Ξ.

To define a uniformly least-favorable submodel, we calculate scores for β and η̄. Let denote the space of elements h = (h₁, h₂) such that h₁ ∈ ℝ^p and h₂ ∈ _η. Consider the one-dimensional submodel defined by the map $t\mapsto {\overline{\psi }}_t\equiv \overline{\psi }+t(h_1,{\int }_0^{(\cdot )}{h}_2\left(u\right)d\overline{\eta }\left(u\right))$, h ∈ . The derivative of log ℓ_n(ψ̄, ζ) with respect to t evaluated at t = 0 yields score operators ℓ̇_n(ψ̄, ζ)(h) ≡ (ℓ̇_nβ(h₁), ℓ̇_nη̄(h₂)), where

$$\begin{array}{l}{\dot{\ell }}_{n\beta }(\overline{\psi },\zeta )(h_1)={\mathbb{P}}_n{\dot{\ell }}_{\beta }(h_1)\hfill \\ ={\mathbb{P}}_n\left\{\delta h_1^{\prime }Z(X)-(1+\delta \zeta )\frac{{\int }_0^{\tau }{h}_1^{\prime }Z(u)Y(u)e^{\beta \prime Z(u)}{(\overline{\eta }(u)+1)}^{\zeta -1}d\overline{\eta }(u)}{1+\zeta {\int }_0^{\tau }{h}_1^{\prime }Z(u)Y(u)e^{\beta \prime Z(u)}{(\overline{\eta }(u)+1)}^{\zeta -1}d\overline{\eta }(u)}\right\},\hfill \end{array}$$

and

$$\begin{array}{l}{\dot{\ell }}_{n\overline{\eta }}\left(\overline{\psi },\zeta \right)\left(h_2\right)={\mathbb{P}}_n{\dot{\ell }}_{\overline{\eta }}\left(h_2\right)\hfill \\ ={\mathbb{P}}_n\{{\int }_0^{\tau }\left(h_2\left(u\right)+\frac{\left(\zeta -1\right){\int }_0^u{h}_2\left(s\right)d\overline{\eta }\left(s\right)}{\overline{\eta }\left(u\right)+1}\right)dN\left(u\right)-\left(1+\delta \zeta \right)\hfill \\ \frac{{\int }_0^{\tau }Y\left(u\right)e^{\beta \prime Z(u)}{\left(\overline{\eta }\left(u\right)+1\right)}^{\zeta -2}[\left(\zeta -1\right){\int }_0^u{h}_2\left(s\right)d\overline{\eta }\left(s\right)+h_2\left(u\right)\left(1+\overline{\eta }\left(u\right)\right)]d\overline{\eta }\left(u\right)}{1+\zeta {\int }_0^{\tau }Y\left(u\right)e^{\beta \prime Z(u)}{\left(\overline{\eta }\left(u\right)+1\right)}^{\zeta -1}d\overline{\eta }\left(u\right)}\},\hfill \end{array}$$

with Y(u) ≡ 1{V ≥ u}.

To obtain the information operator, we consider the two-dimensional submodel defined by the map $(s,t)\mapsto {\overline{\psi }}_{st}\equiv \overline{\psi }+s(h_1,{\int }_0^{(\cdot )}{h}_2(u)d\overline{\eta }(u))+t({\stackrel{\sim }{h}}_1,{\int }_0^{(\cdot )}{\stackrel{\sim }{h}}_2(u)d\overline{\eta }(u))$, where h, h̃ ∈ . Define _∞ = {h ∈ : ‖h‖ < ∞}. The information operator σ̄_θ̄(h) : _∞ ↦ _∞ is given by −P₀∂/∂s∂tℓ₁(ψ̄_st)|_s,t=0 = ψ̄(σ̄_θ̄(h)). We will show σ̄_θ̄ is one-to-one, continuously invertible and onto uniformly over ζ ∈ Ξ, via Part (1) of Lemma 7 in the appendix for which it suffices to show that the information operator for the original parameterization σ_θ is one-to-one, continuously invertible and onto uniformly over ζ ∈ Ξ.

With the same derivation of σ̄_θ̄, σ_θ : _∞ ↦ _∞ takes the form

$${\sigma }_{\theta }(h)=\left(\begin{array}{cc}{\sigma }_{\theta }^{11}& {\sigma }_{\theta }^{12}\\ {\sigma }_{\theta }^{21}& {\sigma }_{\theta }^{22}\end{array}\right)\left(\begin{array}{c}{h}_1\\ h_2\end{array}\right),$$

where

$$\begin{array}{l}{\sigma }_{\theta }^{11}(h_1)=-P_{0}S(\theta ){\int }_0^{\tau }{h}_1^{\prime }Z(u)Y(u)e^{\beta \prime Z(u)}d{\eta }_0(u),\\ {\sigma }_{\theta }^{12}(h_2)=-P_{0}S(\theta ){\int }_0^{\tau }{h}_2(u)Z(u)Y(u)e^{\beta \prime Z(u)}d{\eta }_0(u),\\ {\sigma }_{\theta }^{21}(h_1)=-P_{0}S(\theta )(1+\zeta \eta (T\wedge \tau )(\zeta ))h_1^{\prime }Z(u)Y(u)e^{\beta \prime Z(u)}\\ -\zeta P_{0}S(\theta )Y(u){\int }_0^{\tau }{h}_1^{\prime }Z(u)Y(u)e^{\beta \prime Z(u)}d{\eta }_0(u),\\ {\sigma }_{\theta }^{22}(h_2)=-P_{0}S(\theta )(1+\zeta \eta (T\wedge \tau )(\zeta ))h_2(u)Y(u)e^{\beta \prime Z(u)}\\ -\zeta P_{0}S(\theta )Y(u){\int }_0^{\tau }{h}_2(u)Y(u)e^{\beta \prime Z(u)}d{\eta }_0(u),\end{array}$$

with S(θ) = −(1 + δζ)/(1 + ζη̄(τ))².

All of the operators ${\sigma }_{\theta }^{ij}$, 1 ≤ i, j ≤ 2 are uniformly compact and bounded over ζ ∈ Ξ. With a similar argument as in Kosorok et al. (2004), the linear operator σ_θ : _∞ ↦ _∞ is one-to-one, continuously invertible and onto uniformly over ζ ∈ Ξ by verifying the conditions of Lemma 8 in the appendix. Thus a uniformly least-favorable submodel for estimating β in the presence of η̄ and ζ is η̄_t(β, η̄, ζ) = (1 + (β − t)′v_θ̄) dη̄, where v_θ̄ : ℝ ↦ ℝ^p is the uniformly least-favorable direction at (β₀, η̄, ζ) defined by $h\prime v_{\overline{\theta }}={({\overline{\sigma }}_{\overline{\theta }}^{22})}^{-1}{\overline{\sigma }}_{\overline{\theta }}^{21}h$, h ∈ ℝ^p. This leads to ℓ(t, β, η̄, ζ) = ℓ₁(β, η̄_t(θ), ζ). Because η̄_β(β, η̄, ζ) = η̄, B1 is satisfied. Since ∂/∂t|_t=β0ℓ(t, β₀, η̄₀, ζ) = ℓ̇_β(β₀, ψ̄₀, ζ) = ℓ̃_β(ψ̄₀, ζ), where ℓ̃_β(x) = ℓ̇_β − ℓ̇_η̄v_θ is the efficient score for β, B2 is satisfied due to the continuity of the involved functions with respect to ψ̄ and the fact that Ξ is compact. The efficient information for β is _β = P₀ℓ̃_βℓ̃′_β. That {ℓ̇(t, ψ̄, ζ) : t ∈ U, ψ̄ ∈ V, ζ ∈ Ξ} is P₀-Donsker and {ℓ̈(t, ψ̄, ζ) : t ∈ U, ψ̄ ∈ V, ζ ∈ Ξ} is P₀-Glivenko-Cantelli for some neighborhoods U and V follows from standard empirical process arguments.

It follows from Corollary 8.1.3 in Golub and Van Loan (1983) that the set of eigenvalues is a continuous function of the elements of _β(θ̄), which are continuous functions of ζ. The set of eigenvalues is therefore a continuous function of ζ. Thus inf_ζ λ_min{Ĩ_β(θ₀(ζ))} > 0 by the compactness of Ξ, and Assumption A1 is satisfied.

The consistency of the restricted MLE ${\widehat{\overline{\psi }}}_0$ and the uniform consistency of the unrestricted MLE ${\widehat{\overline{\psi }}}_n(\zeta )$ can be established via the self-consistency equation approach, with arguments similar to the proof of Theorem 3 in Kosorok et al. (2004). We omit the details. To verify the uniform no-bias condition (4), it suffices to show that ${sup}_{\zeta \in \mathrm{\Xi }}{\Vert \widehat{\overline{\eta }}}_{{\widehat{\beta }}_n}(\zeta )-{\overline{\eta }}_0{\Vert }_{\infty }=O_{P_0}^{\star }({\Vert \widehat{\beta }}_n-{\beta }_0\Vert +n^{-1/2})$, for any sequence β̃_n → β₀, where “⋆” denotes the outer expectation. By verifying conditions in Lemma 9 in the appendix, we have

$$\underset{\zeta \in \mathrm{\Xi }}{sup}({\mathbb{P}}_n-P_0)\left\{{\dot{\ell }}_{\overline{\psi }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )-{\dot{\ell }}_{\overline{\psi }}({\beta }_0,{\overline{\eta }}_0,\zeta )\right\}=o_{P_0}^{\star }(n^{-1/2}).$$

Together with the fact that ${\mathbb{P}}_n{\dot{\ell }}_{\overline{\psi }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )=P_0{\dot{\ell }}_{\overline{\psi }}({\beta }_0,{\overline{\eta }}_0,\zeta )=0$, we obtain $P_0\left\{{\dot{\ell }}_{\overline{\psi }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )-{\dot{\ell }}_{\overline{\psi }}({\beta }_0,{\overline{\eta }}_0,\zeta )\right\}=P_0{\dot{\ell }}_{\overline{\psi }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )-{\mathbb{P}}_n{\dot{\ell }}_{\overline{\psi }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )=-({\mathbb{P}}_n-P_0){\dot{\ell }}_{\overline{\psi }}({\beta }_0,{\overline{\eta }}_0,\zeta )+o_{P_0}^{\star }(n^{-1/2})$, uniformly over ζ ∈ Ξ.

Let l̇_ψ(h) ≡ (l̇_β(h₁), l̇_η(h₂)) denote the score operator of ψ with the original parameterization. It was shown in Kosorok et al. (2004) that the operator ψ ↦ l̇_ψ is Fréchet differentiable with derivative ψ(σ_θ(h)), and it can be strengthened to uniform Fréchet differentiability due to the smoothness of the involved functions. Since φ_ζ is uniformly Fréchet differentiable, by Part (2) of Lemma 7, the chain rule for uniform Fréchet differentiability ℓ̇_ψ̄ ≡ (ℓ̇_β, ℓ̇_η̄) is uniformly Fréchet differentiable with derivative ${\sigma }_{{\phi }_{\zeta }^{-1}(\overline{\theta })}\circ {\dot{\phi }}_{\zeta }^{-1}(\overline{\theta })$.

By the uniform Fréchet differentiability of ${\dot{\ell }}_{\overline{\psi }},{\overline{\sigma }}_{\overline{\theta }}({\stackrel{\sim }{\beta }}_n,{\widehat{\overline{\eta }}}_{{\overline{\beta }}_n}(\zeta )-{\overline{\eta }}_0)=P_0\left\{{\dot{\ell }}_{\overline{\psi }}({\widehat{\beta }}_n,{\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta ),\zeta )-{\dot{\ell }}_{\overline{\psi }}({\beta }_0,{\overline{\eta }}_0,\zeta )\right\}+o_{P_0}^{\mathrm{\Xi }}(\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +\Vert {\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta )-{\overline{\eta }}_0{\Vert }_{\infty })$. Since σ̄_θ̄ is linear, the first term on the right-hand side is of the order O_P0(n^−1/2). It follows that ${sup}_{\zeta \in \mathrm{\Xi }}\Vert {\widehat{\overline{\eta }}}_{{\stackrel{\sim }{\beta }}_n}(\zeta )-{\overline{\eta }}_0{\Vert }_{\infty }=O_{P_0}^{\star }(\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +n^{-1/2})$, since σ̄_θ̄ is uniformly continuously intrtible over ζ ∈ Ξ.

5. Simulation results

This section presents simulation results regarding the finite-sample properties of the proposed optimal test statistics for Example 2, the change-point Cox model with current status data. The simulation study was designed with several objectives. First, we demonstrate how to compute the asymptotic critical values with the proposed weighted bootstrap procedure. Second, we analyze the empirical type I error of the proposed tests and compare with the nominal size of the tests. Third, we compare the power of the optimal tests with that of other tests such as the sup score statistics (equivalent to the likelihood ratio statistic) and some naive (pointwise) tests under several different alternatives. Fourth, we evaluate the sensitivity of the power of the optimal test to the choice of c under several different alternatives.

A single time-independent covariate Z with a uniform [0,1] distribution was used. The threshold covariate Y = Z. The parameter α was set at α₀ = 0, with the cumulative baseline hazards A₀(t) = 3t². The censoring time was uniformly distributed on the interval [0,5]. This resulted in a censoring rate of about 25% under the null hypothesis. Under the alternative, we set β₁₀ = −0.5, β₂₀ ∈ {−0.3, −0.5, −0.8}. The range of β₂₀ values reflects the distance from the null. We consider the following alternative distributions of ζ:

1. The weight J(ζ) degenerates to one point at 0.5, that is ζ = 0.5.

2. A uniform weight J(ζ) with support on [0.05, 0.95].

For all the scenarios, we compute the optimal tests with a uniform weight on [0.05,0.95]. The sample size for each simulated data set was 300. For each simulated data set, 250 bootstraps were generated with standard exponential weights truncated at 5, to compute the critical values for R_n(ζ), the naive score statistic at several ζ values, sup_ζ R_n(ζ), the sup score statistic and ER_n, the weighted exponential score statistic. We take c = 0, 0.5, 1, 3 and ∞ respectively. Each scenario was replicated 1000 times. To compute the restricted MLE under the null, we use the iterative convex minorant algorithm. Empirical type I error and power results for selected subsets of the test statistics described above are provided in Table 1.

Table 1.

The empirical type I error and power of the proposed tests, sample size n = 300, 1000 simulations, with bootstrap size 250. The worst case Monte Carlo error for table entries is 0.016. The Monte Carlo error is 0.007 and 0.009 for empirical type I error with nominal size 0.05 and 0.1 respectively. The empirical power results are based on size 0.05 tests.

Empirical type I error
Nominal size	Weighted exponential tests, c=					Sup score	Naive tests Rn(ζ), ζ =
	0	0.5	1	3	∞		0.3	0.9	0.5
	J(ζ) ∼ Uniform[0.05, 0.95]
0.05	0.056	0.057	0.045	0.063	0.046	0.058	0.043	0.044	0.039
0.10	0.098	0.103	0.109	0.095	0.099	0.085	0.112	0.103	0.100
Empirical power
True alternative	Weighted exponential tests, c=					Sup score	Naive tests Rn(ζ), ζ =
	0	0.5	1	3	∞		0.3	0.9	0.5
	J(ζ) ∼ Uniform[0.05, 0.95]
ζ=0.5
η = −0.3	0.646	0.647	0.653	0.653	0.656	0.688	0.243	0.044	0.692
η = −0.5	0.835	0.833	0.839	0.845	0.847	0.865	0.616	0.076	0.840
η = −0.8	0.922	0.925	0.928	0.928	0.928	0.968	0.957	0.174	0.942
J(ζ) ∼ Uniform[0.05, 0.95]
η = −0.3	0.320	0.320	0.320	0.320	0.312	0.211	0.133	0.055	0.142
η = −0.5	0.485	0.488	0.492	0.494	0.500	0.405	0.258	0.083	0.272
η = −0.8	0.748	0.757	0.763	0.768	0.769	0.605	0.494	0.183	0.413

We now make several general comments on the simulation results. The empirical type I error for all the tests is quite close to the nominal level. When the alternative distribution of ζ is correctly specified, the optimal test is notably more powerful than the sup score statistic and naive tests. When the true alternative distribution of ζ degenerates to one point, although the weighted exponential tests are no longer optimal, the empirical powers are still superior to the naive tests with misspecified ζ. We also observe that the empirical power of the sup score statistics is comparable to that of the naive test at the true ζ, which may be due to the fast convergence rate of the change-point estimator. For all the alternatives considered, the empirical power of the weighted exponential tests seems to increase as c increases. However, the trend is rather weak. In many cases, the difference in power is less than 0.01. This suggests that the direction of the test (specifically, least favorable curve in this paper), rather than the scale of the curve, is most critical for the power of the weighted exponential test.

6. Discussion

In this paper, we consider tests of hypotheses when the parameters are not identifiable under the null in semiparametric models. Our optimality results apply to a large class of semiparametric testing problems under loss of identifiability, where nuisance parameters may not be root-n estimable either under the null or alternative. We note that our current regularity conditions are not directly applicable for testing under loss of inevitability when the parameter of interest is not root-n estimable. One example is testing for homogeneity in mixture models, where the usual first order Taylor approximation may not be possible (Chen et al., 2004; Chernoff and Lander, 1995; Dacunha-Castelle and Gassiat, 1999; Lindsay, 1995; Liu and Shao, 2003). A higher order expansion is required. Although not directly covered by our framework, the homogeneity tests may possess a uniform quadratic expansion (Zhu and Zhang, 2006), thus permitting a generalization of our results to general quadratic expansions. In the following, we conclude the paper with a brief discussion of this generalization.

To be concrete, let us consider a two-component mixture with density g(ρ, μ₁, μ₂, η) = ρf(μ₁, η) + (1 − ρ)f(μ₂, η), where f(μ, η) is a parametric p.d.f. with parameters μ ∈ ℝ^p, η ∈ ℝ^q, such as a location-scale family. Let β = μ₂ − μ₁, θ = (ρ, β′, ${\mu }_1^{\prime }$, η′) ′, and the hypothesis of interest is β = 0; that is, there is only a single component in the mixture. For convenience, we assume the mixing proportion ρ ∈ (0, 1] and μ₁ = μ₂ = μ₀ under the null.

In this example, ρ is not identifiable and μ₁ and μ₂ are mutually indistinguishable under the null. Simple algebra shows that the information matrix for ψ ≡ (β, μ₁) is singular under the null, for arbitrary values of ρ, which corresponds to the fact that μ₁ and μ₂ are not root-n estimable (Chen and Chen, 2003; Zhu and Zhang, 2004). We consider the following reparameterization: μ̄ = (1 − ρ)(μ₁ − μ₀) + ρ(μ₂ − μ₀), and v̄ = (1 − ρ)(μ₁ − μ₀)² + ρ(μ₂ − μ₀)², which can be considered as “mixed mean” and “mixed variance”. Let β ≡ (μ̄, v̄) and ψ̄ ≡ (μ̄, v̄, η). We can establish the identifiability of ψ̄ and the consistency and the root-n rate of the MLE of ψ̄ under the null. Furthermore, under a set of assumptions on the parameter space (e.g. the cone condition in Andrews, 1999 and 2001) and the stochastic differentiability and equicontinuity of the involved functions, we can establish the following quadratic expansion of the loglikelihood w.r.t. $\overline{\psi }:L_n(\overline{\psi },\zeta )=L_n({\overline{\psi }}_0,\zeta )+{(\overline{\psi }-{\overline{\psi }}_0)}^{\prime }{S}_{n\zeta }({\overline{\psi }}_0)+\frac{1}{2}{(\overline{\psi }-{\overline{\psi }}_0)}^{\prime }{B}_{\zeta }({\overline{\psi }}_0)(\overline{\psi }-{\overline{\psi }}_0)+r_n(\zeta )$, where $r_n(\zeta )=o_p^{\mathrm{\Xi }}(1)$, and S_nζ(·) and B_ζ(·) are different from but similar in structure to the score and information processes for ψ̄ indexed by ζ.

When the nuisance parameter η is not present, a similar weight as in the current paper for ψ̄ can be chosen as Q_ζ(·) = B_ζ(·). The corresponding weighted exponential tests are still optimal in the Neyman-Pearson sense. If η is present, a uniformly least favorable curve for this quadratic expansion with respect to β would need to be characterized. This is beyond the scope of the current paper but is an interesting topic for future research.

7. Proofs

Proof of Lemma 1. Since φ̇_ζ is linear, continuously invertible and one-to-one, the tangent set for η and η̄ are identical. By the chain rule, ${\dot{\ell }}_{\overline{\eta }}(\gamma )={\dot{l}}_{\eta }{\dot{\phi }}_{\zeta }^{-1}(\gamma )$ for any γ in the tangent set of η̄. The efficient score for β with the parameter (β, η, ζ) is: ${\stackrel{\sim }{l}}_{\beta }(\beta ,\eta ,\zeta )=(I-{\dot{l}}_{\eta }{({\dot{l}}_{\eta }^{\star }{l}_{\eta })}^{-1}{\dot{l}}_{\eta }^{\star }){\dot{l}}_{\beta }(\psi ,\zeta )$ and with the parameter (β, η̄, ζ) is: $(I-{\dot{\ell }}_{\overline{\eta }}{({\dot{\ell }}_{\overline{\eta }}^{\star }{\ell }_{\overline{\eta }})}^{-1}{\dot{\ell }}_{\overline{\eta }}^{\star }){\dot{\ell }}_{\beta }(\psi ,\zeta )$. The efficient score function is invariant under such reparameterizations since

$$\begin{array}{l}I-{\dot{\ell }}_{\overline{\eta }}{({\dot{\ell }}_{\overline{\eta }}^{\star }{\ell }_{\overline{\eta }})}^{-1}{\dot{\ell }}_{\overline{\eta }}^{\star }(\overline{\psi },\zeta )=I-{\dot{l}}_{\eta }{\dot{\phi }}_{\zeta }^{-1}{({\dot{\phi }}_{\zeta }^{-1\star }{\dot{l}}_{\eta }^{\star }{\dot{l}}_{\eta }{\dot{\phi }}_{\zeta }^{-1})}^{-1}{\dot{\phi }}_{\zeta }^{\star }{\dot{l}}_{\eta }^{\star }(\psi ,\zeta )\\ =I-{\dot{l}}_{\eta }{\dot{\phi }}_{\zeta }^{-1}{\dot{\phi }}_{\zeta }{({\dot{l}}_{\eta }^{\star }{\dot{l}}_{\eta })}^{-1}{\dot{\phi }}_{\zeta }^{\star }{({\dot{\phi }}_{\zeta }^{\star })}^{-1}{\dot{l}}_{\eta }^{\star }(\psi ,\zeta )=I-{\dot{l}}_{\eta }{({\dot{l}}_{\eta }^{\star }{l}_{\eta })}^{-1}{\dot{l}}_{\eta }^{\star }(\psi ,\zeta ),\end{array}$$

and ${\dot{\ell }}_{\beta }(\overline{\psi },\zeta )={\dot{l}}_{\beta }(\psi ,\zeta )$. That the efficient information matrix is invariant under reparameterizations thus follows from its definition.

Proof of Lemma 2. It suffices to show that under the full rank reparameterization, for any random sequence β̃_n →_P0 β₀,

$$\begin{array}{l}logp{\ell }_n({\stackrel{\sim }{\beta }}_n,\zeta )=logp{\ell }_n({\beta }_0,\zeta )+n({\stackrel{\sim }{\beta }}_n-{\beta }_0)\prime {\mathbb{P}}_n{\stackrel{\sim }{\ell }}_{\beta }({\overline{\psi }}_0,\zeta )\\ -\frac{1}{2}n({\stackrel{\sim }{\beta }}_n-{\beta }_0)\prime {\stackrel{\sim }{ℐ}}_{\beta }({\overline{\psi }}_0,\zeta )({\widehat{\beta }}_n-{\beta }_0)+o_{P_0}^{\mathrm{\Xi }}{(\sqrt{n}\Vert {\stackrel{\sim }{\beta }}_n-{\beta }_0\Vert +1)}^2.\end{array}$$ (11)

By Assumption B2, B4 and the dominated convergence theorem, for every $(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }})-({\beta }_0,{\beta }_0,{\overline{\eta }}_0)\to 0$, we have $P_0{(\dot{\ell }(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }},\zeta )-{\stackrel{\sim }{\ell }}_{\beta }({\overline{\psi }}_0,\zeta ))}^2=o^{\mathrm{\Xi }}(1)$. Similarly, we have $P_0\ddot{\ell }(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }},\zeta )-P_0\ddot{\ell }({\beta }_0,{\beta }_0,{\overline{\eta }}_0,\zeta )=o^{\mathrm{\Xi }}(1)$. The derivative of the function t ↦ log ℓ(t, ψ̄₀, ζ) satisfies P₀ℓ̈(β₀, ψ̄₀, ζ) = −_β(ψ̄₀, ζ). These facts, together with the empirical process conditions, imply that for every random sequence $(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }})\to ({\beta }_0,{\beta }_0,{\overline{\eta }}_0),{\mathbb{G}}_n\dot{\ell }(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }})-{\mathbb{G}}_n{\stackrel{\sim }{\ell }}_{\beta }({\overline{\psi }}_0,\zeta )=o_{P_0}^{\mathrm{\Xi }}(1)$ and ${\mathbb{P}}_n\ddot{\ell }(\stackrel{\sim }{t},\stackrel{\sim }{\beta },\stackrel{\sim }{\overline{\eta }},\zeta )+{\stackrel{\sim }{ℐ}}_{\beta }({\overline{\psi }}_0,\zeta )=o_{P_0}^{\mathrm{\Xi }}(1)$. The subsequent steps of the proof are similar to those used in the proof of Theorem 1 in Murphy and van der Vaart (2000), and we omit the details.

Proof of Theorem 1. The proof takes several steps. We first show the asymptotic equivalence of these statistics, which is summarized in Lemma 5 below. With a small abuse of notation, let ${\mathit{\text{PLR}}}_n\equiv \int p{l}_n(\beta +h/\sqrt{n},\zeta )d{Q}_{\zeta }(h)dJ(\zeta )/p{l}_n({\beta }_0,\zeta )$. This is the profile likelihood ratio of the alternative over the null and it can be approximated by

$$\begin{array}{l}{\overline{\mathit{\text{PLR}}}}_n\equiv \int exp\left\{\frac{1}{2}{\overline{\beta }}_n({\theta }_0(\zeta ))\prime {\stackrel{\sim }{I}}_{\beta }({\theta }_0(\zeta )){\overline{\beta }}_n({\theta }_0(\zeta ))\right\}\\ \times \int exp\left\{-\frac{1}{2}({\overline{\beta }}_n({\theta }_0(\zeta ))-h)\prime {\stackrel{\sim }{I}}_{\beta }({\theta }_0(\zeta ))({\overline{\beta }}_n({\theta }_0(\zeta ))-h)\right\}d{Q}_{\zeta }(h)dJ(\zeta ),\end{array}$$

with the linear statistic ${\overline{\beta }}_n({\theta }_0(\zeta ))\equiv \sqrt{n}{\stackrel{\sim }{I}}_{\beta }^{-1}({\theta }_0(\zeta )){\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta ))$. An approximate exponential Wald statistic ${\overline{EW}}_n$ is defined as

$${\overline{EW}}_n={(1+c)}^{-\frac{p}{2}}\int exp\left(\frac{1}{2}\frac{c}{1+c}{\overline{W}}_n(\zeta )\right)dJ(\zeta ),$$

where ${\overline{W}}_n(\zeta )={\overline{\beta }}_n{({\theta }_0(\zeta ))}^{\prime }{\stackrel{\sim }{I}}_{\beta }({\theta }_0(\zeta )){\overline{\beta }}_n({\theta }_0(\zeta ))$.

Now we show the asymptotic distribution of these tests under the null hypothesis. Assume without loss of generality that β̂_n and ${\widehat{\overline{\psi }}}_{\mathrm{n}}$ take their values in U and V as defined in Assumption B4, respectively. Following Lemma 3.2 in Murphy and van der Vaart (1997), we have ${\mathbb{G}}_n(\dot{\ell }({\widehat{\beta }}_n,{\widehat{\overline{\psi }}}_n,\zeta )-{\stackrel{\sim }{\ell }}_{\beta }({\overline{\psi }}_0,\zeta )){\to }_{P_0}0$. Thus ℓ̃_β(ψ₀, ζ) = ℓ̃_β(θ₀(ζ)) is P₀–Donsker as a class indexed by ζ ∈ Ξ and ${\overline{EW}}_n{\to }_{d}e\chi (c)$ by the continuous mapping theorem. Lemma 5 below then gives the desired results of Theorem 1.

Lemma 5. Under the null hypothesis and Assumptions A–C, (1) ${\mathit{\text{PLR}}}_n-{\overline{\mathit{\text{PLR}}}}_n{\to }_{P_0}0$, (2) ${\overline{\mathit{\text{PLR}}}}_n={\overline{\mathit{\text{EW}}}}_n$, (3) ${\overline{\mathit{\text{EW}}}}_n-{\mathit{\text{EW}}}_n{\to }_{P_0}0$, (4) ${\overline{\mathit{\text{EW}}}}_n-{\mathit{\text{ER}}}_n{\to }_{P_0}0$ and (5) ER_n − ELR_n →_P0 0.

Proof. For notational simplicity, let β̄_n = β̄_n(θ₀(ζ)) and Ĩ₀ = Ĩ_β(θ₀(ζ)). We first show (1). For 0 < M < ∞, define

$${\mathit{\text{PLR}}}_n(M)={\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\Vert h\Vert \le M}p{l}_n({\beta }_0+h/\sqrt{n},\zeta )d{Q}_{\zeta }(h)dJ(\zeta )/p{l}_n({\beta }_0,\zeta ),$$

and

$$\begin{array}{l}{\overline{\mathit{\text{PLR}}}}_n(M)={\int }_{\zeta \in \mathrm{\Xi }}exp\left(\frac{1}{2}{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_{\beta }{\overline{\beta }}_n\right)\hfill \\ \times {\int }_{\Vert h\Vert \le M}exp\left(-\frac{1}{2}{({\overline{\beta }}_n-h)}^{\prime }{\stackrel{\sim }{I}}_0({\overline{\beta }}_n-h)\right)d{Q}_{\zeta }(h)dJ(\zeta ).\hfill \end{array}$$

Note that for any M > 0,

$$\begin{array}{l}\left|{\mathit{\text{PLR}}}_n-{\overline{\mathit{\text{PLR}}}}_n\right|\le \left|{\mathit{\text{PLR}}}_n-{\mathit{\text{PLR}}}_n(M)\right|+\left|{\mathit{\text{PLR}}}_n(M)-{\overline{\mathit{\text{PLR}}}}_n(M)\right|\\ +\left|{\overline{\mathit{\text{PLR}}}}_n-{\overline{\mathit{\text{PLR}}}}_n(M)\right|.\end{array}$$

Hence it suffices to show that (i) |PLR_n − PLR_n(M)| →_P0 0, (ii) $|{\overline{\mathit{\text{PLR}}}}_n-{\overline{\mathit{\text{PLR}}}}_n(M)|{\to }_{P_0}0$ and (iii) $|{\mathit{\text{PLR}}}_n(M)-{\overline{\mathit{\text{PLR}}}}_n(M)|{\to }_{P_0}0$, as n → ∞ and ∀M : 0 < M < ∞. To show (i), for any ε > 0,

$$\mathit{\text{Pr}}(|{\mathit{\text{PLR}}}_n-{\mathit{\text{PLR}}}_n(M)|>\epsilon )\le {\epsilon }^{-1}{P}_0|{\mathit{\text{PLR}}}_n-{\mathit{\text{PLR}}}_n(M)|$$

$$={\epsilon }^{-1}P{\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\Vert h\Vert >M}\frac{p{l}_n({\beta }_0+h/\sqrt{n},\zeta )}{p{l}_n({\beta }_0,\zeta )}d{Q}_{\zeta }(h)dJ(\zeta )$$ (12)

$$\le {\epsilon }^{-1}P{\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\Vert h\Vert >M}\frac{p{l}_n({\widehat{\beta }}_n(\zeta ),(\zeta )}{p{l}_n({\beta }_0,\zeta )}d{Q}_{\zeta }(h)dJ(\zeta )$$ (13)

$$\to {\epsilon }^{-1}P{\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\Vert h\Vert >M}(1+o_p(1))d{Q}_{\zeta }(h)dJ(\zeta )$$ (14)

$$={\epsilon }^{-1}{\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\Vert h\Vert >M}d{Q}_{\zeta }(h)dJ(\zeta )+o(1),$$ (15)

where (12) uses Assumption C and (13) holds by definition of the profile likelihood. (14) holds by Assumption B3 and Lemma 2. (15) holds by Fubini's theorem. The right hand side of (15) can be made arbitrarily small for all n by taking M large enough, since Q_ζ is a uniformly tight measure.

For (ii), we have

$$\begin{array}{l}|{\overline{\mathit{\text{PLR}}}}_n-{\overline{\mathit{\text{PLR}}}}_n(M)|\\ ={\int }_{\zeta \in \Xi }exp\left(\frac{1}{2}{\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }{({\theta }_0(\zeta ))}^{\prime }{\stackrel{\sim }{I}}_0^{-1}{\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta ))\right)\\ \times {\int }_{\Vert h\Vert >M}exp\left(-\frac{1}{2}{({\overline{\beta }}_n-h)}^{\prime }{\stackrel{\sim }{I}}_0({\overline{\beta }}_n-h)\right)d{Q}_{\zeta }(h)dJ(\zeta )\\ \le exp\left(\frac{1}{2}\underset{\zeta \in \Xi }{\text{sup}}\Vert {\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta )){\Vert }^2\underset{\zeta \in \Xi }{\text{sup}}\Vert {\stackrel{\sim }{I}}_0^{-1}\Vert \right)\times {\int }_{\zeta \in \Xi }{\int }_{\Vert h\Vert >M}d{Q}_{\zeta }(h)dJ(\zeta ).\end{array}$$ (16)

In the inequality, ${\Vert {\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta ))\Vert }^2=O_{P_0}^{\mathrm{\Xi }}(1)$ follows from Assumption B4. The fact that $\Vert {\stackrel{\sim }{I}}_0^{-1}\Vert =O_{P_0}^{\mathrm{\Xi }}(1)$ follows from Assumption A. The last term ∫_{ζ ∈Ξ} ∫_‖h‖>M dQ_ζ(h)dJ(ζ) → 0, as M → ∞. Hence (16)=o_p(1), as M → ∞.

Now we show (iii). For contiguous sequences ${\beta }_0+h/\sqrt{n}{\to }_{P_0}{\beta }_0$ and ‖h‖ ≤ M, Lemma 2 yields the following expansion of the profile likelihood under the null:

$$\begin{array}{l}log{pl}_n({\beta }_0+h/\sqrt{n,}\zeta )=log{pl}_n({\beta }_0,\zeta )+\sqrt{n}{h}^{\prime }{\mathbb{P}}_n{\stackrel{\sim }{l}}_{\beta }({\theta }_0(\zeta ))-\frac{1}{2}{h}^{\prime }{\stackrel{\sim }{I}}_{0}h+o_{P_0}^{\mathrm{\Xi }}(1)\\ =\frac{1}{2}{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_0{\overline{\beta }}_n-\frac{1}{2}{({\overline{\beta }}_n-h)}^{\prime }{\stackrel{\sim }{I}}_0({\overline{\beta }}_n-h)+o_{P_0}^{\mathrm{\Xi }}(1),\end{array}$$

therefore,

$$\begin{array}{l}{\mathit{\text{PLR}}}_n(M)={\iint }_{\Vert h\Vert \le M}({pl}_n({\beta }_0+h/\sqrt{n},\zeta )-{pl}_n({\beta }_0,\zeta )){dQ}_{\zeta }(h)dJ(\zeta )\\ ={\iint }_{\Vert h\Vert \le M}exp(\frac{1}{2}{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_0{\overline{\beta }}_n-\frac{1}{2}{({\overline{\beta }}_n-h)}^{\prime }{\stackrel{\sim }{I}}_0({\overline{\beta }}_n-h)+o_p^{\mathrm{\Xi }}(1)){dQ}_{\zeta }(h)dJ(\zeta )\\ ={\overline{\mathit{\text{PLR}}}}_n(M)+o_p(1),\end{array}$$

where the last equality follows from ${\overline{\mathit{\text{PLR}}}}_n(M)=O_p(1)$, by arguments analogous to those used in (16) above. The proof for Part (1) is now completed.

For Part (2), since $h\sim Q_{\zeta }=N(0,{c\stackrel{\sim }{I}}_0^{-1})$,

$${\overline{\mathit{\text{PLR}}}}_n={\int }_{\zeta \in \mathrm{\Xi }}{\xi }_n(\zeta )dJ(\zeta ),$$

with

$$\begin{array}{ll}\hfill {\xi }_n(\zeta )=& \int exp\left(\frac{1}{2}{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_0{\overline{\beta }}_n-\frac{1}{2}{({\overline{\beta }}_n-h)}^{\prime }{\stackrel{\sim }{I}}_0({\overline{\beta }}_n-h)\right)d{Q}_{\zeta }(h)\hfill \\ \hfill =& {(2\pi )}^{-p/2}{\text{det}}^{1/2}({\stackrel{\sim }{I}}_0/c)\hfill \\ \hfill & \times \int exp\left[\frac{1}{2}\left\{{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_0\overline{{\beta }_n}-(h-{\overline{\beta }}_n)\prime {\stackrel{\sim }{I}}_0(h-{\overline{\beta }}_n)-\frac{h^{\prime }{\stackrel{\sim }{I}}_{0}h}{c}\right\}\right]dh\hfill \\ \hfill =& {(1+c)}^{-p/2}exp\left(\frac{1}{2}\frac{c}{1+c}{\overline{\beta }}_n^{\prime }{\stackrel{\sim }{I}}_0{\overline{\beta }}_n\right),\hfill \end{array}$$

where the last equality holds by integrating out a normal density.

For Part (3), it follows from Lemma 2 and Assumption B3 that $\sqrt{n}\Vert {\widehat{\beta }}_n(\zeta )-{\beta }_0\Vert =O_{P_0}^{\mathrm{\Xi }}(1)$, and reapplication of Lemma 2 and the argmax theorem yields $\sqrt{n}({\widehat{\beta }}_n(\zeta )-{\beta }_0)={\stackrel{\sim }{I}}_{\beta }{({\theta }_0(\zeta ))}^{-1}\sqrt{n}{\mathbb{P}}_n{\stackrel{\sim }{I}}_{\beta }({\theta }_0(\zeta ))+o_{P_0}^{\mathrm{\Xi }}(1)$. Part (3) now follows.

For the proof of Part (4) and Part (5), it suffices to show that $W_n(\zeta )-R_n(\zeta )=o_{P_0}^{\mathrm{\Xi }}(1)$ and $R_n(\zeta )-L{R}_n(\zeta )=o_{P_0}^{\mathrm{\Xi }}(1)$. These results follow from Donsker properties and standard arguments. We omit the details. The proof of Lemma 5 is thus complete.

Proof of Corollary 1. The proof is similar to the proof of Theorem 1. We omit the details.

Proof of Corollary 2. The proof follows the same lines as the proof of Part (2) (iii) of Lemma 5, with

$$\begin{array}{l}W(q_{\zeta },\zeta )={(2\pi )}^{-p/2}{\text{det}}^{1/2}\left(\frac{1+c}{c}{\stackrel{\sim }{I}}_0\right)\times \\ \int exp\left[-\frac{1+c}{2c}{(\mathrm{\lambda }-\frac{c}{1+c}{\overline{\beta }}_n)}^{\prime }{\stackrel{\sim }{I}}_0(\mathrm{\lambda }-\frac{c}{1+c}{\overline{\beta }}_n)-{\mathrm{\lambda }}^{\prime }{\langle q_{\zeta }-{\stackrel{\sim }{q}}_{\zeta },P{\dot{l}}_{\eta }^{\star }{\dot{l}}_{\eta }{(q_{\zeta }-{\stackrel{\sim }{q}}_{\zeta })}^{\prime }\rangle }_{\eta }\mathrm{\lambda }\right]d\mathrm{\lambda },\end{array}$$ (17)

where det is the determinant of a matrix, 〈·, ·〉_η is the inner product defined on _η, and W(q_ζ, ζ) ≤ 1 since ${\langle q_{\zeta }-{\stackrel{\sim }{q}}_{\zeta },p{\dot{l}}_{\eta }^{\star }{\dot{l}}_{\eta }{(q_{\zeta }-{\stackrel{\sim }{q}}_{\zeta })}^{\prime }\rangle }_{\eta }$ is nonnegative definite.

Lemma 6. Under Assumptions A–C, the densities ${\ell }_n({\overline{\psi }}_0+h/\sqrt{n},\zeta )$ and $\int {\ell }_n({\overline{\psi }}_0+h/\sqrt{n},\zeta )d{Q}_{\zeta }(h)dJ(\zeta )$ are contiguous to the densities $l_n^0$. As a consequence, the results of Lemma 5 still hold under local alternatives $\{P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }\}$ and $\{\int P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }d{Q}_{\zeta }(h)dJ(\zeta )\}$.

Proof. Assumption C implies that a LAN (local asymptotic normal) expansion for the log-likelihood ratio holds immediately by Lemma 3.10.11 of van der Vaart and Wellner (1996):

$${\mathrm{\Lambda }}_{n\zeta }\equiv log\left(\frac{d{P}_{{\psi }_0+h/\sqrt{n},\zeta }^n}{d{P}_{{\psi }_0,\zeta }^n}\right)=\frac{1}{\sqrt{\mathrm{n}}}\text{∑}_{i=1}^n{A}_{\zeta }h(X_i)-\frac{1}{2}{\Vert A_{\zeta }h\Vert }^2+o_{P_0}(1).$$

It follows from LAN that Λ_nζ →_d W_ζ, where W_ζ ∼ N(−1/2‖A_ζh‖², ‖A_ζh‖²), under P₀. Therefore, under P₀,

$$exp({\mathrm{\Lambda }}_{n\zeta })\equiv \frac{d{P}_{{\overline{\psi }}_0+h\sqrt{n},\zeta }^n}{d{P}_0^n}{\to }_{d}exp{W}_{\zeta .}$$

P₀(exp(W_ζ)) = 1, using the formula for the moment generating function of the normal distribution. By Le Cam's first Lemma (van der Vaart, 1996, page 88), we conclude that the sequences of probability measures $\{P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }\}$ and {P₀} are contiguous, for every ζ ∈ Ξ. Consequently the convergence in probability that hold under P₀ also hold under $\{P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }\}$ and vice-versa. Similarly, since P(eχ) = 1 using the formula for the moment generating function of the χ² distribution, we conclude that the sequences $\{\int P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }^{n}d{Q}_{\zeta }(h)dJ(\zeta )\}$ and $P_0^n$ are contiguous.

Proof of Theorem 2. We define a $\sqrt{n}$-neighborhood of β₀ as a collection of sequences ${\beta }_n(h_{\beta })={\beta }_0+h_{\beta }/\sqrt{n}+o(n^{-1/2})$, for h_β ∈ ℝ^p. A $\sqrt{n}$ neighborhood of η is similarly defined as ${\eta }_n(h_{\eta })=\eta +h_{\eta }/\sqrt{n}+o(n^{-1/2})$, for h_η ∈ _η. With a minor abuse of notation, a local form of the hypotheses can be written as:

$$H_0:\overline{\psi }={\overline{\psi }}_0\text{vs.}{H}_{\text{1}}:\overline{\psi }={\overline{\psi }}_0+h_1/\sqrt{n},$$ (18)

where h₁ ∈ ℝ^p × _η takes the value (h_β1, h_η1), with $h_{\eta 1}={\stackrel{\sim }{q}}_{\zeta }^{\prime }{h}_{\beta 1}$. We note that the least favorable direction q̃_ζ is invariant under the choice of φ_ζ, as a consequence, the contiguous alternative H₁ is also invariant under the choice of φ_ζ.

Define

$$L{R}_n\equiv \frac{\int {\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )}{{\ell }_n^0}.$$ (19)

A test defined by LR_n is

$${\stackrel{\sim }{\gamma }}_n=\{\begin{array}{c}1,\text{if}L{R}_n>{\stackrel{\sim }{k}}_{\alpha n},\\ {\stackrel{\sim }{\mathrm{\lambda }}}_n,\text{if}L{R}_n={\stackrel{\sim }{k}}_{\alpha n},\\ 0,\text{if}L{R}_n<{\stackrel{\sim }{k}}_{\alpha n},\end{array}$$

where k̃_αn > 0, λ̃_n ∈ [0, 1] are constants such that the rejection probability is α under the null. For notational simplicity, let $P_1^n=\int P_{{\overline{\psi }}_0+h_1/\sqrt{n},\zeta }^{n}d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )$. By the Neyman-Pearson lemma, for all n ≥ 1 and any test φ_n with level α, with a minor abuse of notation,

$$\begin{array}{l}\underset{n\to \infty }{\text{lim}}\int {\phi }_n\left\{\int {\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )\right\}d{P}_1^n\\ \le \underset{n\to \infty }{\text{lim}}\int {\stackrel{\sim }{\gamma }}_n\left\{\int {\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )\right\}d{P}_1^n\end{array}$$ (20)

$$=\underset{n\to \infty }{\text{lim}}\int I(L{R}_n>{\stackrel{\sim }{k}}_{\alpha n})\left\{\int {\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )\right\}d{P}_1^n$$ (21)

$$=\underset{n\to \infty }{\text{lim}}\int \left\{\int I({\mathit{\text{PLR}}}_n>{\stackrel{\sim }{k}}_{\alpha n})d{P}_{{\overline{\psi }}_0+h_1/\sqrt{n},\zeta }^n\right\}d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )$$ (22)

$$=\underset{n\to \infty }{\text{lim}}\int \left\{\int I(E{W}_n>{\stackrel{\sim }{k}}_{\alpha n})d{P}_{{\overline{\psi }}_0+h_1/\sqrt{n},\zeta }^n\right\}d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta ),$$ (23)

where (21) follows since LR_n has an absolutely continuous asymptotic distribution under the contiguous alternative H₁ and by Fubini's theorem. (22) follows since PLR_n − LR_n = o_P (1) under H₁, which will be established at the end of the proof. (23) follows from Lemma 6. The results for ER_n and ELR_n also follow from Lemma 6. By Fubini's theorem, we obtain $\text{lim}{\text{sup}}_{n\to \infty }\int \left\{{\phi }_n(P_{{\overline{\psi }}_0+h_1/\sqrt{n},\zeta }^n)\right\}d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )\le {\text{lim}}_{n\to \infty }\int \left\{\int I(E{W}_n>{\stackrel{\sim }{k}}_{{\alpha }_n})d{P}_{{\overline{\psi }}_0+h_1/\sqrt{n},\zeta }^n\right\}d{Q}_{\zeta }(h_{\beta 1})dJ(\zeta )$, which implies that the proposed tests have the greatest weighted average power asymptotically in the class of all tests of asymptotic significance level α, against the alternative $P_{{\overline{\psi }}_0+h/\sqrt{n},\zeta }^n$.

To show PLR_n − LR_n = o_P (1) under H₁, it suffices to show PLR_n − LR_n = o_P (1) under the null by Lemma 6. Define $L{R}_n(M)\equiv {\int }_{\zeta \in \mathrm{\Xi }}{\int }_{\left|\right|h\left|\right|\le M}{\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )d{Q}_{\zeta }(h)dJ(\zeta )/{\ell }_n({\overline{\psi }}_0,\zeta )$, and note that ∀M : 0 < M < ∞, |PLR_n − LR_n| ≤ |PLR_n − PLR_n(M)| + |PLR_n(M) − LR_n(M)| + |LR_n − LR_n(M)|. Hence it suffices to show that (i) |PLR_n − PLR_n(M)| →_P0 0, (ii) $|{\overline{\mathit{\text{LR}}}}_n-{\overline{\mathit{\text{LR}}}}_n(M)|{\to }_{P_0}0$ and (iii) $|{\mathit{\text{PLR}}}_n(M)-{\overline{\mathit{\text{LR}}}}_n(M)|{\to }_{p_0}0$, as n → ∞. Part (i) was shown in Lemma 5. Part (ii) can be similarly established by taking M large enough and using Assumption A.

To show Part (iii), we take Taylor expansion of $log{\ell }_n({\overline{\psi }}_0+h_1/\sqrt{n},\zeta )$ at (ψ̄₀, ζ) with respect to h_β along the direction q̃_ζ, which leads to the following expansion in the least favorable submodel:

$$log{\ell }_n({\overline{\psi }}_0+\frac{h_1}{\sqrt{n}},\zeta )=log{\ell }_n({\overline{\psi }}_0,\zeta )+\sqrt{n}{h}_{\beta 1}^{\prime }{\mathbb{P}}_n\dot{\ell }({\beta }_0,{\overline{\psi }}_0,\zeta )+\frac{1}{2}{h}_{\beta 1}^{\prime }{\mathbb{P}}_n\ddot{\ell }(\stackrel{\sim }{\beta },\stackrel{\sim }{\psi },\zeta )h_{\beta 1}.$$

On the right-hand side, we can replace ℙ_nℓ̇(β₀, ψ̄₀, ζ) by ${\mathbb{P}}_n{\dot{\ell }}_{\beta }({\overline{\psi }}_0,\zeta )+o_{P_0}^{\mathrm{\Xi }}(1)$, and ℙ_nℓ̈(β̃, ψ̃, ζ) by $-{\stackrel{\sim }{I}}_{\beta }({\overline{\psi }}_0,\zeta )+o_{P_0}^{\mathrm{\Xi }}(1)$, according to Assumption B2. Comparing the above display and Lemma 2 with ${\stackrel{\sim }{\beta }}_n\equiv h_{\beta 1}/\sqrt{n}$, we obtain Part (iii).

Proof of Theorem 3. The equivalence of the three tests under local alternatives is shown in Lemma 6. To show their asymptotic distribution, a key step is to establish that β̄_n converges under $P_{{\overline{\psi }}_0+h/\sqrt{n},{\zeta }_1}^n$ in distribution to the process ζ ↦ (θ₀(ζ)) + v_⋆ (h_β, ζ, ζ₁), where v_⋆ (h_β, ζ, ζ₁) ≡ P₀l̃_β(θ₀(ζ))l̃_β(θ₀(ζ₁))′ h_β, by Theorem 3.10.12 in van der Vaart and Wellner (1996). The result follows by Lemma 6 and the continuous mapping theorem.

Proof of Theorem 4. The equivalence of the three tests under local alternatives is shown in Lemma 6. Since the sequences of densities $\int {\ell }_n({\stackrel{\leftharpoonup }{\Psi }}_0+h/\sqrt{n},\zeta )d{Q}_{\zeta }(h)dJ(\zeta )$ are contiguous to the density $l_n^0$, we have

$$\left({\mathit{\text{ELR}}}_n,\frac{\int {dP}_{{\overline{\Psi }}_0+h/\sqrt{n},\zeta }^{n}d{Q}_{\zeta }\left(h\right)dJ\left(\zeta \right)}{{dP}_0^n}\right){⇝}_d\left(e\chi \left(c\right),e\chi \left(c\right)\right),$$

under P₀. Then ELR_n →_d rχ(c) under $\int {dP}_{{\overline{\Psi }}_0+h/\sqrt{n},\zeta }^{n}d{Q}_{\zeta }\left(h\right)dJ\left(\zeta \right)$, by Le Cam's third lemma.

Proof of Lemma 3. The proof mainly involves an argument that for an arbitrary, possibly random sequence {ζ_n}, the distance between the minimizer of the Kullback-Leibler information and θ̂_n(ζ_n) goes to zero. Consequently, the assertion of Lemma 3 follows from the arbitrariness of the sequence ζ_n and Slutsky's theorem. We omit the details.

Proof of Lemma 4. The proof mainly involves a uniform “peeling device” with an adaptation of the proof of Theorem 3.2 given in Murphy and van der Vaart (1999), which details we omit.

Lemma 7. (1) Assume φ_ζ : _φ ⊂ ↦ _ψ ⊂ is one-to-one, continuously invertible and onto and ψ_ζ : ψ ⊂ ↦ is one-to-one, continuously invertible and onto, then ψ_ζ ○ φ_ζ : _φζ ↦ is one-to-one, continuously invertible and onto. (2). Assume φ_ζ : _φ ⊂ ↦ _ψ ⊂ is uniformly Fréchet differentiable at θ ∈ _ψ and ψ_ζ : _ψ ⊂ ↦ is uniformly Fréchet differentiable at φ_ζ(θ) over ζ ∈ Ξ. Then ψ_ζ ∘ φ_ζ : _φ ↦ is uniformly Fréchet differentiable at θ with derivative ${\psi }_{\zeta }^{\prime }({\phi }_{\zeta }(\theta ))\circ {\phi }_{\zeta }^{\prime }(\theta )$.

Proof. For Part (1), it suffices to note that ‖ψ̇_ζ(φ_ζ(θ))(φ̇_ζ(θ)(h))‖ ≥ c₁‖φ̇_ζ(θ)(h)‖ ≥ c₁c₂‖h‖. For Part (2), we note that, ψ_ζ ∘ φ_ζ(θ + th) − ψ_ζ ∘ φ_ζ(θ) = ψ_ζ(φ_ζ(θ)+tk_t) − ψ_ζ(φ_ζ(θ)), where k_t = {φ_ζ(θ+th) − φ_ζ(θ)}/t. So we rewrite the uniform Fréchet difference as ψ_ζ(φ_ζ(θ + h))(·) − ψ_ζ(φ_ζ(θ))(·) = ψ̇_ζ(φ_ζ(θ))(φ_ζ(θ+h) − φ_ζ(θ))+o ^Ξ(‖φ_ζ(θ+h) − φ_ζ(θ)‖) = ψ̇_ζ(φ_ζ(θ))ψ̇_ζ(θ)(h) + o^Ξ(‖h‖).

Lemma 8. Let A_ζ = T_ζ + K_ζ : ↦ be a linear operator between Banach spaces, where T_ζ is onto and there exists c₁ > 0, such that ‖T_ζh‖ ≥ c₁‖h‖ for all h ∈ and ζ ∈ Ξ, and K_ζ is uniformly compact, i.e., {ζ ∈ Ξ, ‖h‖ ≤ 1 : ∪_‖h‖≤1K_ζh} is compact. Then if N(A_ζ) = {0} for all ζ ∈ Ξ, then A_ζ is onto and there exists c₂ > 0 such that ‖A_ζh‖ ≥ c₂‖h‖, ∀ζ ∈ Ξ and all h ∈ .

Proof. Since, for an arbitrary random sequence ζ_n, $T_{{\zeta }_n}^{-1}$ is continuous, the operator $T_{{\zeta }_n}^{-1}K:\mathbb{\text{E}}\mapsto \mathbb{\text{D}}$ is compact. Hence $I+T_{{\zeta }_n}^{-1}{K}_{{\zeta }_n}$ is one-to-one and therefore also onto be a result of Riesz for compact operators. Thus T_ζn +K_ζn is also onto. We will be done if we can show that $I+T_{{\zeta }_n}^{-1}{K}_{{\zeta }_n}$ is continuously invertible, since that would imply that ${(T_{{\zeta }_n}+K_{{\zeta }_n})}^{-1}={(I+T_{{\zeta }_n}^{-1}{K}_{{\zeta }_n})}^{-1}{T}_{{\zeta }_n}^{-1}$ is bounded. The remainder of the proof follows the proof of Lemma 6.17 in Kosorok (2008).

Lemma 9. Suppose that U_n(ψ, ζ)(h) = ℙ_nv(ψ, ζ)(h) and U(ψ, ζ)(h) = Pv(ψ, ζ)(h) for given P-measurable functions v(ψ, ζ)(h) indexed by Ψ × Ξ and an arbitrary index set _η. Assume ψ = ψ₀, v(ψ₀, ζ)(h) = v(ψ₀)(h). If ${\widehat{\psi }}_n\left(\zeta \right)={\psi }_0+o_P^{\mathrm{\Xi }}(1)$, the class of functions {v(ψ, ζ)(h) − v(ψ₀)(h) : ‖ψ−ψ₀‖ < δ, h ∈ _η, ζ ∈ Ξ} is P−Donsker for some δ > 0 and sup_{ζ∈Ξ,h∈η} P₀(v(ψ, ζ)(h)−v(ψ₀)(h))² → 0, as ψ → ψ₀, then ${\text{sup}}_{\zeta \in \mathrm{\Xi }}\Vert \sqrt{n}(U_n-U)({\widehat{\psi }}_n(\zeta ),\zeta )-\sqrt{n}(U_n-U)({\psi }_0,\zeta )\Vert =o_P^{\star }(1+\sqrt{n}\Vert {\widehat{\psi }}_n(\zeta )-{\psi }_0\Vert )$.

Proof. This is a “uniform” version of Lemma 3.3.5 in van der Vaart and Wellner (1996). Let Ψ_δ ≡ {ψ : ‖ψ − ψ₀‖ < δ} and define an extraction function f : ℓ^∞(Ψ_δ × Ξ × _η) × Ψ_δ ↦ ℓ^∞(_η × Ξ) as f(z, ψ, ζ)(h) ≡ z(ψ, ζ, h), where z ∈ ℓ^∞(Ψ_δ × _η × Ξ). Since f is continuous at every point (z, ψ₁, ζ), we have sup_{h∈η,ζ∈Ξ} |z(ψ, ζ, h) − z(ψ₁, ζ, h)| → 0 as ψ → ψ₁. Define the stochastic process Z_n(ψ, ζ, h) = _n(v(ψ, ζ)(h) − v(ψ₀, ζ)(h)) indexed by Ψ_δ × Ξ × _η. By assumptions, Z_n converges weakly in ℓ^∞(Ψ_δ × Ξ × _η) to a tight Gaussian process Z₀ with continuous sample paths with respect to the metric ρ_ζ defined by ${\rho }_{\zeta }^2$((ψ₁, ζ, h₁), (ψ₂, ζ, h₂)) = P(v(ψ₁, ζ)(h₁) − v(ψ₀, ζ)(h₁) − v(ψ₂, ζ)(h₂) + v(θ₀, ζ)(h₂))², at fixed ζ. Since as assumed, sup_{h∈η,ζ∈Ξ} ρ_ζ((ψ, h), (ψ₀, h)) → 0, we have that f is continuous at almost all sample paths of Z₀ uniformly over ζ ∈ Ξ. The result now follows by Slutksy's theorem and the continuous mapping theorem.