Maximum Likelihood Estimations and EM Algorithms with Length-biased Data

SUMMARY

Length-biased sampling has been well recognized in economics, industrial reliability, etiology applications, epidemiological, genetic and cancer screening studies. Length-biased right-censored data have a unique data structure different from traditional survival data. The nonparametric and semiparametric estimations and inference methods for traditional survival data are not directly applicable for length-biased right-censored data. We propose new expectation-maximization algorithms for estimations based on full likelihoods involving infinite dimensional parameters under three settings for length-biased data: estimating nonparametric distribution function, estimating nonparametric hazard function under an increasing failure rate constraint, and jointly estimating baseline hazards function and the covariate coefficients under the Cox proportional hazards model. Extensive empirical simulation studies show that the maximum likelihood estimators perform well with moderate sample sizes and lead to more efficient estimators compared to the estimating equation approaches. The proposed estimates are also more robust to various right-censoring mechanisms. We prove the strong consistency properties of the estimators, and establish the asymptotic normality of the semi-parametric maximum likelihood estimators under the Cox model using modern empirical processes theory. We apply the proposed methods to a prevalent cohort medical study. Supplemental materials are available online.

1. INTRODUCTION

When the observed failure times are not randomly selected from the target population of interest but with probability proportional to their underlying length, we have length-biased time-to-event data. Length-biased data are naturally encountered in applications of renewal processes (Cox and Miller, 1977; Vardi, 1982; Dewanji and Kalbfleisch, 1987; Vardi, 1989), industrial applications (Kvam, 2008), etiologic studies (Simon, 1980), genome-wide linkage studies (Terwilliger et al., 1997), epidemiologic cohort studies (Keiding, 1991; Gail and Benichou, 2000; Gordis, 2000; Sansgiry and Akman, 2000; Scheike and Keiding, 2006), cancer prevention trials (Zelen and Feinleib, 1969; Zelen, 2004), and studies of labor economy (Lancaster, 1990; McClean and Devine, 1995; De Uña Álvarez et al., 2003). In observational studies, a prevalent cohort design that draws samples from individuals with a condition or disease at the time of enrollment is generally more efficient and practical. The recruited patients who have already experienced an initiating event are followed prospectively for the failure event (e.g. disease progression or death) or are right censored. Under this sampling design, individuals with longer survival times measured from the onset of the disease are more likely to be included in the cohort, whereas those with shorter survival times are selectively excluded. Length-biased sampling thereby manifests in the observations, because the “observed” time intervals from initiation to failure within the prevalent cohort tend to be longer than those arising from the underlying distribution of the general population. How to properly adjust for potential selection bias in analyzing length-biased data has been a longstanding statistical problem. Although we use a prevalent cohort study in medical applications here to illustrate length-biased data, it is apparent that the issues caused by biased sampling are common in many potential applications and sampling designs.

In a seminal paper, Vardi (1989) described the multiplicative censorship model, which connected four well-investigated statistical problems: A. Estimating a non-parametric distribution function under multiplicative censoring, B. Estimating the underlying distribution in renewal processes, C. Solving a nonparametric deconvolution problem, and D. Estimating a monotone decreasing density function. Vardi (1989) presented problems A and C, which have a natural connection with the measurement error problem and inverse problem discussed by van der Vaart and Wellner (1992) and Bickel and Ritov (1994). Most importantly, Vardi (1989) and Wang (1991) showed that the nonparametric maximum likelihood estimation (NPMLE) of the survival distribution under multiplicative censoring (problem B) is equivalent to the nonparametric estimation for survival distribution of the observed length-biased data. The large sample properties of the corresponding NPMLE is established in Asgharian et al. (2002), and the asymptotic efficiency of the NPMLE follows from Asgharian and Wolfson (2005) and van der Vaart (1998, Theorem 25.47). In this paper we explore the potential to extend the approach of Vardi (1989) to nonparametric estimations in more general settings and to semiparametric regression models.

The Cox proportional hazards model, the most popular semiparametric model for regression analysis of traditional survival data, assumes a nonparametric baseline hazard function and a regression function of the covariates (Cox, 1972, 1975). Only limited literature exists that describes modeling risk factors on the distribution of the underlying population when observed failure times are subject to length bias. Recently, Tsai (2009) generalized the pseudo-partial likelihood approach of Wang (1996) to model right-censored length-biased data. Qin and Shen (2010) proposed inverse weighted estimating equation approaches for right-censored length-biased data under the proportional hazards model. These approaches do not provide a straightforward way to analyze length-biased data if the censoring time depends on the covariates, and may not yield efficient estimators. For traditional survival data, Zeng and Lin (2007) demonstrated that estimating equations approaches under either the semiparametric Cox model or the transformation models are less efficient than the profile maximum likelihood estimation approach. (For related works on the profile likelihood for traditional survival data, see Nielsen et al. (1992); Klein (1992); Murphy (1994, 1995); Murphy and van der Vaart (2000); Zeng et al. (2005); Zeng and Lin (2007).) For right-censored length-biased data, we expect the similar efficiency advantage of the maximum likelihood estimation (MLE) method, and the robustness of the method to various assumptions of the censoring distribution.

Implementing the profile likelihood method is much more challenging when working with right-censored length-biased data compared to traditional survival data. One significant difference is that the full profile likelihoods have positive support on both censored and failure time points for length-biased data in contrast to the MLEs for traditional survival data and the conditional likelihood estimates for length-biased data. We propose new expectation-maximization (EM) algorithms for the maximum likelihood estimation of the nonparametric and semiparametric Cox regression models for right-censored length-biased data. One new aspect of our method is that we derive the likelihood for the unobserved (i.e. left-truncated) subpopulation given the observed length-biased data in the full likelihood, which serves as the missing data mechanism in the EM algorithm. In constrast to the EM algorithm of Vardi (1989), which estimates the underlying distribution function via estimation of the biased distribution, our EM algorithm directly estimates the target unbiased distribution function. As a result, any model and parameter constraints for the target distribution function can be directly imposed.

The rest of the paper is organized as follows. In Section 2, we introduce a new EM algorithm for the nonparametric estimation of the target distribution given length-biased data. In Section 3, we apply the new EM algorithm to estimate a distribution function with an increasing failure rate constraint. In Section 4, we propose the maximum semiparametric likelihood estimation under the Cox proportional hazards model and derive the large sample properties for length-biased data. We provide a convenient profile estimation approach based on the EM algorithm, with which the standard software for Cox regression can be adapted for right-censored length-biased data. We describe our simulation studies in Section 5 and the application of our method to a data example in Section 6. Section 7 contains some concluding remarks.

2. A NEW EM ALGORITHM FOR ESTIMATING NONPARAMETRIC SURVIVAL FUNCTION

Consider a prevalent cohort study in which the subjects are diagnosed with a disease and are at risk for a failure event. Let T̃, be the duration from the disease onset to failure with the unbiased density function f(t) = dF(t)/dt and survival function S(t). The observed data include the backward recurrence time A (from disease onset to the study entry), forward recurrence time V (from the study entry to failure), and length-biased time T = A+V. Based on the renewal theory (Vardi, 1982, 1989; Lancaster, 1990, Chapter 3), the joint distribution of (A, V) is

$$\frac{f(a+v)}{\mu },a,v>0,\text{where}\mu =\int tf(t)dt.$$

When the prevalent cohort is followed prospectively, V is subject to right censoring. The censoring time, denoted by C, is measured from the study entry. Let δ = I(V < C) be the censoring indicator and assume that (A, V) is independent of C. Let X = min(A + V, A + C). Denote the observed data as (X_i, A_i, δ_i), i = 1, 2, …, n. The density function of the observed biased T is defined as g(y) = dG(y)/dy, where dG(y) = ydF(y)/μ, and the survival function of T̃ is

$$S(t)={\int }_t^{\infty }dF(u)=\mu {\int }_t^{\infty }{u}^{-1}dG(u).$$

Therefore, the likelihood for the observed data (X_i, A_i, δ_i) is proportional to

$$\prod _{i=1}^n\frac{f^{{\delta }_i}(X_i)S^{1-{\delta }_i}(X_i)}{\mu }\propto \prod _{i=1}^n{[dG(X_i)]}^{{\delta }_i}\prod _{i=1}^n{\left[{\int }_{x\ge X_i}{x}^{-1}dG(x)\right]}^{1-{\delta }_i}.$$ (1)

Vardi (1989) proposed an EM-algorithm for the NPMLE of G. Using the relationship between G and F, dF(t) = t⁻¹dG(t)/∫t⁻¹dG(t), the NPMLE for F can be derived. However, it is often difficult to impose constraints on F when F is estimated from the NPMLE of G, because the constraints on F may not be easily translated to the constraints on G.

As demonstrated in Vardi (1989), to maximize (1) it is sufficient to consider the discrete version of distribution F, i.e., p(T̃ = t_i) = p_i, nonparametrically on the point masses at

$$t_1<t_2<\dots <t_k,$$

where t₁, …, t_k are the ordered unique failure and censoring times for {X₁, …, X_n}, k ≤ n. In principle, the length-biased observations (A, T) can be equivalently generated from a truncation model with

$$A\sim U(0,\widehat{\tau }),\stackrel{\sim }{T}\sim F,\text{on}(0,\widehat{\tau }),$$ (2)

where τ̂ = t_k, A and T̃ are independent, dF(t_i) = p_i and ${\sum }_{i=1}^k{p}_i=1$, and (A, T) is observed if and only if T̃ ≥ A. The probability of observing a length-biased observation under this setting is π = P(T̃ ≥ A) = E(T̃)/τ̂.

We propose an EM algorithm with a different missing mechanism to directly estimate the target distribution, F. For a cohort subject to left truncation, the biased samples on n subjects denoted by O = {(X₁, δ₁, A₁), ···, (X_n, δ_n, A_n), A_i ≤ X_i, i = 1, ···, n}, are observed, whereas the data on m subjects are left truncated. Here the latent left-truncated data are denoted by ${\mathit{O}}^{\ast }=\{(T_1^{\ast },A_1^{\ast }),\dots ,(T_m^{\ast },A_m^{\ast }),A_i^{\ast }>T_i^{\ast },i=1,2,\dots ,m\}$. The random integer m then follows a negative binomial distribution with parameter π. The probability mass function of m is

$$\left(\begin{array}{c}m+n-1\\ m\end{array}\right){\left(1-\pi \right)}^m{\pi }^n,m=0,1,2,\dots \text{and}E(m\mid \mathit{O})=n(1-\pi )/\pi .$$

Following the principle of the EM algorithm, we think of {O, O*} as the ‘complete data’, and consider that pseudo missing data also referred to as “ghosts” data in Turnbull (1976) are ${\mathit{O}}^{\ast }=\{(T_1^{\ast },A_1^{\ast }),\dots ,(T_m^{\ast },A_m^{\ast }),m\}$ and the observed ‘incomplete data’ are O. We derive the full likelihood including the component of the truncated observations. The log-likelihood based on the complete data {O, O*} is

$$\sum _{j=1}^k\left[\sum _{i=1}^{n}I(T_i=t_j)+\sum _{i=1}^{m}I(T_i^{\ast }=t_j)\right]log{p}_j,$$ (3)

where T_i ≥ A_i, i = 1, 2, ···, n and $T_l^{\ast }<A_l^{\ast }$, l = 1, ···, m. Then conditional on the observed data,

$$\begin{array}{l}E\left[\sum _{i=1}^{n}I(T_i=t_j)|\mathit{O}\right]=\sum _{i=1}^n\left\{{\delta }_{i}I(T_i=t_j)+(1-{\delta }_i)P(T_i=t_j\mid T_i\ge A_i,T_i\ge X_i)\right\}\\ =\sum _{i=1}^n\left[{\delta }_{i}I(X_i=t_j)+(1-{\delta }_i)\frac{I(X_i\le t_j)p_j}{{\int }_{s\ge X_i}f(s)ds}\right],\end{array}$$

because (1 − δ_i)P (T_i = t_j|T_i ≥ A_i, T_i ≥ X_i) = (1 − δ_i)P (T = t_j | T ≥ X_i) = (1 − δ_i)I(X_i ≤ t_j)p_i/S(X_i). Conditional on the observed data O, the expectation for the missing left-truncated data can be expressed as

$$E\left\{E\left[\sum _{i=1}^{m}I(T_i^{\ast }=t_j)|m\right]|\mathit{O},T^{\ast }<A^{\ast }\right\}.$$

Under the truncation model specified in (2),

$$EI(T^{\ast }=t_j|\mathit{O},T^{\ast }<A^{\ast })=Pr(T^{\ast }=t_j,A^{\ast }>t_j)/Pr(T^{\ast }<A^{\ast })=\frac{p_j(1-t_j/\widehat{\tau })}{1-\pi }.$$

This together with E(m | O) = n(1 − π)/π,

$$E\left\{E\left[\sum _{i=1}^{m}I(T_i^{\ast }=t_j)|m\right]|\mathit{O},T^{\ast }<A^{\ast }\right\}=\frac{n(1-\pi )}{\pi }\frac{(1-t_j/\widehat{\tau })p_j}{1-\pi }=\frac{n}{\pi }(1-t_j/\widehat{\tau })p_j.$$

Subject to ${\sum }_{i=1}^k{p}_i=1$ and p_i ≥ 0, we maximize the expected complete-data log-likelihood conditional on the observed data via the EM algorithm,

$${\ell }_E(\mathit{p})=\sum _{j=1}^k{w}_{j}log{p}_j,$$ (4)

where p = (p₁, ···, p_k), and

$$w_j=\sum _{i=1}^n\left[{\delta }_{i}I(X_i=t_j)+(1-{\delta }_i)\frac{p_{j}I(X_i\le t_j)}{{\sum }_{j=1}^k{p}_{j}I(X_i\le t_j)}\right]+\frac{n}{\pi }(1-t_j/\widehat{\tau })p_j.$$

By simple algebra, ${\sum }_{j=1}^k{w}_j=n+n(1-\pi )/\pi =n/\pi$. The following iterative EM algorithm can be used to solve p̂_j for j = 1, ···, k.

• Step 1Select an arbitrary $p_j^{(0)}$ satisfying ${\sum }_{j=1}^k{p}_j^{(0)}=1,p_j^{(0)}\ge 0$.
• Step 2
  Solve $p_j^{(1)}$ by maximizing (4), so that we replace $p_j^{(0)}$ with

  $${\widehat{p}}_j^{(1)}=\frac{{\widehat{\pi }}^{(0)}}{n}\left\{\sum _{i=1}^n\left[{\delta }_{i}I(X_i=t_j)+(1-{\delta }_i)\frac{{\widehat{p}}_j^{(0)}I(X_i\le t_j)}{{\sum }_{j=1}^k{\widehat{p}}_j^{(0)}I(X_i\le t_j)}\right]+\frac{n}{{\widehat{\pi }}^0}(1-\frac{t_j}{\widehat{\tau }}){\widehat{p}}_j^{(0)}\right\},$$ (5)

  where ${\widehat{\pi }}^{(0)}={\sum }_{j=1}^k{t}_j{\widehat{p}}_j^{(0)}/\widehat{\tau }$.

With a given convergence criterion, we can solve p_j iteratively. Let p̂_j denote the MLE of p_j, j = 1, ···, k, the NPMLE $\widehat{F}(t)={\sum }_{j=1}^k{\widehat{p}}_{j}I(t_j\le t),\widehat{\pi }=\int td\widehat{F}(t)/\widehat{\tau }$, π̂ = ∫ tdF̂(t)/τ̂, and

$$Q_1^n(t)=\frac{1}{n_1}\sum _{i=1}^n{\delta }_{i}I(X_i\le t),Q_0^n(t)=\frac{1}{n_0}\sum _{i=1}^n(1-{\delta }_i)I(X_i\le t),$$

where $n_1={\sum }_{i=1}^n{\delta }_i$ and $n_0={\sum }_{i=1}^n(1-{\delta }_i)$. Thus, the limiting form of (5) is

$$d\widehat{F}(t)=\widehat{\pi }{n}_1{n}^{-1}{dQ}_1^n(t)+\widehat{\pi }{n}_0{n}^{-1}d\widehat{F}(t){\int }_0^t\frac{{dQ}_0^n(s)}{1-\widehat{F}(s)}+(1-t/\widehat{\tau })d\widehat{F}(t).$$ (6)

Remark 1

In contrast to the NPMLE for traditional survival analyses, which has jumps only at the observed failure time points, the proposed NPMLE for length-biased data has jumps at all observed but unique points including censored times, similar to that of Vardi (1989).

Remark 2

Equation (6) for the constructed EM algorithm with the unbiased distribution function F is equivalent to that for Vardi’s EM algorithm based on a ‘multiplicative-censorship’ model with the biased distribution function G. Denoting dĜ(t) = tdF̂(t)/μ̂, where μ̂ = π̂τ̂, we re-express equation (6) as an equation of Ĝ,

$$d\widehat{G}(t)=n_1{n}^{-1}{dQ}_1^n(t)+n_0{n}^{-1}\frac{d\widehat{G}(t)}{t}{\int }_0^t{\left[{\int }_{r\ge s}{r}^{-1}d\widehat{G}(r)\right]}^{-1}{dQ}_0^n(s),$$

which is the same equation derived by Vardi (1989) and Vardi and Zhang (1992). The advantage of the new EM algorithm is that it directly estimates the target distribution function of the unbiased data, which allows one to directly impose constraints on F. This advantage will be further elucidated in the next two sections.

Remark 3

The ‘missing’ data (i.e. left-truncated failure times), { $T_1^{\ast },\cdots ,T_m^{\ast }$} are assumed not subject to right censoring. It is clear that whether T* is subject to right censoring or not is irrelevant in the derivation of the above EM algorithm.

Remark 4

The development of the methods and large sample properties is focused on [0, τ] throughout the paper, where τ is a finite upper bound of the support of the population survival times, and Λ(τ) < ∞. In practice, τ can be estimated by $t_k={max}_{i=1}^n{X}_i$. We prove (in Appendix A.2) the following lemma that τ̂ ≡ t_k → τ in probability, and that the convergence rate is faster than n^1/2.

Lemma 1

Suppose that E(C) > 0 and τ < ∞. Then for 1 > η > 1/2, n^η(τ̂ − τ) =o_p(1).

3. NONPARAMETRIC MAXIMUM LIKELIHOOD ESTIMATION WITH INCREASING FAILURE RATE

In some applications, it is known or assumed that the survival function for the target population has an increasing failure rate (Barlow and Proschan, 1975; Padgett and Wei, 1980; Tsai, 1988). The maximum likelihood estimation of a distribution function with an increasing failure rate was derived for traditional right-censored data by Padgett and Wei (1980), and for left-truncated and right-censored data by Tsai (1988). Using the same notation as in Section 2, the observed right-censored length-biased data are denoted by (X, A, δ). Let λ(t) denote the hazard function for the target cumulative density function F. Let z₁ < ··· < z_k* denote the distinct ordered failure times {X₁, ···, X_n}. Let the size of the risk set at time x be denoted by $R(x)={\sum }_{i=1}^{n}I(A_i\le x\le X_i)$ and the size of failure at time x be denoted by $d(x)={\sum }_{i=1}^{n}I(X_i=x,{\delta }_i=1)$. Under the increasing failure rate constraint, Tsai (1988) proposed a maximum conditional likelihood estimator of λ, conditional on the truncation time A,

$$\widehat{\lambda }(y)=\{\begin{array}{ll}0\hfill & y<z_1,\hfill \\ {\widehat{\lambda }}_j\hfill & z_j\le y<z_{j+1};j=1,2,\dots ,k^{\ast }-1,\hfill \\ {\widehat{\lambda }}_{k^{\ast }}\hfill & z_{k^{\ast }}\le y\hfill \end{array}$$

where

$${\widehat{\lambda }}_j=\underset{1\le r\le j}{max}\underset{j\le s\le k^{\ast }-1}{min}\left\{\sum _{i=r}^{s}d(z_i)/\sum _{i=r}^{s}R(z_i)(z_{i+1}-z_i)\right\}.$$

By applying the new EM algorithm, we consider a full likelihood estimation of the hazard function for the target population. Define λ(t_j) = λ_j; p_j can be expressed as ${\lambda }_{j}exp(-{\int }_0^{t_j}\lambda (s)ds)$; thus the expected complete-data log-likelihood function in (4) is

$${\ell }_E(\mathit{\lambda })=\sum _{i=1}^k{w}_{i}log{\lambda }_i-\sum _{i=1}^k{w}_i\sum _{j=1}^i{\int }_{t_{j-1}}^{t_j}\lambda (t)dt,$$ (7)

where λ = (λ₁, ··· λ_k), and t₁ < ··· < t_k is defined in §2. Because the hazard function λ (.) increases with time,

$${\ell }_E(\mathit{\lambda })\le \sum _{i=1}^k{w}_{i}log{\lambda }_i-\sum _{j=1}^k\left[\sum _{i=j}^k{w}_i(t_j-t_{j-1})\right]{\lambda }_{j-1},$$

where λ₀ = 0. Taking a partial derivative of λ_j to the right side of the above inequality, we have

$$\frac{w_j}{{\lambda }_j}-\sum _{i=j+1}^k{w}_i(t_{j+1}-t_j)=0.$$ (8)

Using arguments similar to those of Marshall and Proschan (1965) and Padgett and Wei (1980), the solution to equation (8) also maximizes the expected log-likelihood ℓ_E(λ) defined in (7),

$${\lambda }_j=\frac{w_j}{{\sum }_{i=j+1}^k{w}_i(t_{j+1}-t_j)}.$$

Applying the pool-adjacent-violators algorithm, we can then achieve monotonicity for the NPMLE of λ(·),

$$\widehat{\lambda }(x)=\{\begin{array}{ll}0\hfill & x<t_1\hfill \\ {\widehat{\lambda }}_j\hfill & t_j\le x<t_{j+1};j=1,2,\dots ,k-1,\hfill \\ {\widehat{\lambda }}_k\hfill & x=t_k,\hfill \end{array}$$

where ${\widehat{\lambda }}_j={max}_{1\le r\le j}{min}_{j\le s\le k-1}\left\{{\sum }_{i=r}^s{w}_i/{\sum }_{i=r}^s[{\sum }_{l=i+l}^k{w}_l](t_{i+1}-t_i)\right\}$. Although the formula for the proposed NPMLE of the monotone hazard function bears some similarity to that of Tsai (1988), the full likelihood approach is essentially different from the conditional likelihood approach of Tsai (1988), where the estimated function has jumps only at distinct failure time points. By using the information of the left-truncated data in the full likelihood function, the NPMLE is expected to be more efficient and smoother than the maximum conditional likelihood estimate. We will further compare the two approaches in empirical studies.

As the hazard function λ (t) is increasing on t ∈ [0, τ], the corresponding cumulative hazard function Λ(t) is convex. Let Λ̂_n(·) denote the estimator obtained by the EM algorithm together with the pool-adjacent-violators algorithm. Then Λ̂_n(·) is the greatest convex minorant of Λ_n(·), where Λ_n(·) is the NPMLE Λ(·) when there is no constraint on its shape. The strong consistent results of Λ_n(·) uniformly on [0, τ] can be easily derived from the uniform consistency of its survival function, established by Asgharian and Wolfson (2005). The consistency of Λ̂_n can be inferred, because the pool-adjacent-violator algorithm leads to a continuous map for Λ_n(·) to Λ̂_n(·). The technical details are provided in the Appendix A.3.

4. MLE UNDER COX REGRESSION MODEL

4.1 Full Likelihood and Score Functions

Since the cornerstone work of Cox (1972, 1975), the proportional hazards model has become the standard regression model for analyzing traditional right-censored survival data. Specifically, the covariate-specific hazard function is specified as

$${\lambda }_{\mathit{Z}}(t)=\lambda (t)exp({\mathit{\beta }}^{\prime }\mathit{Z})$$

where Z is a covariate vector and the baseline hazard function λ(t) is not specified parametrically. Breslow (1972) showed that by inserting an estimator (Breslow’s estimator) of the hazard function with a fixed β into the full likelihood, the profile likelihood for β is reduced to Cox’s partial likelihood for β. Later, Kalbfleisch and Prentice (1973) and Andersen et al. (1992) proved that the rank-based likelihood method is also equivalent to the partial likelihood method. When survival data are subject to biased sampling, neither the Cox’s partial likelihood approach nor Kalbfleisch and Prentice’s rank-based likelihood method can be directly applied. This is because the observed biased data do not follow the proportional hazards model that is assumed for unbiased data from the target population; and because the rank-based likelihood method is not applicable, due to the dependency of the length-biased data on the magnitude of the length.

The density function of an unbiased T̃ given Z is denoted by f(t | Z) and the corresponding survival function by S(t | Z). For random but length-biased samples of n subjects, the observed data consist of { ≡ (A_i, X_i, δ_i, Z_i), i = 1, ···, n}, which are n i.i.d. copies of ≡ (A, X, δ, Z). The full likelihood function of the observed data is proportional to

$$L_n=\prod _{i=1}^n\frac{f^{{\delta }_i}(X_i\mid {\mathit{Z}}_i)S^{(1-{\delta }_i)}(X_i\mid {\mathit{Z}}_i)}{{\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)}$$ (9)

where ${\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)={\int }_0^{\infty }tf(t\mid {\mathit{Z}}_i)dt={\int }_0^{\infty }S(t\mid {\mathit{Z}}_i)dt$. The identifiability of the model can be established, similar to the case for the Cox model under traditional survival data, where the identifiability has been established (Elbers and Ridder, 1982). By decomposing the full likelihood to the product of the conditional likelihood of X given A and the marginal likelihood of A, we have

$$L_n=\left[\prod _{i=1}^n\frac{f^{{\delta }_i}(X_i\mid {\mathit{Z}}_i)S^{1-{\delta }_i}(X_i\mid {\mathit{Z}}_i)}{S(A_i\mid {\mathit{Z}}_i)}\right]\left[\prod _{i=1}^n\frac{S(A_i\mid {\mathit{Z}}_i)}{{\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)}\right].$$

Although the estimating equation derived from the likelihood conditional on A (the first component in L_n) shares the same advantage of Cox’s partial likelihood by canceling the baseline hazard function (Wang et al., 1993; Kalbfleisch and Lawless, 1991), the conditional likelihood approach is generally less efficient than the full likelihood approach.

Using the notation of counting process, we denote N_i(t) = I(X_i ≤ t)δ_i, Y_i(t) = I(X_i ≥ t) for i = 1, ···, n. The log-likelihood function of (9) can be expressed as

$${\ell }_n(\mathit{\beta },\mathrm{\Lambda })=n^{-1}\sum _{i=1}^n\left\{{\int }_0^{\tau }\left({\mathit{\beta }}^{\prime }{\mathit{Z}}_i+log\lambda (t)\right){dN}_i(t)-{\mathrm{\Lambda }}_{{\mathit{Z}}_i}(X_i)-log{\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)\right\},$$ (10)

where $\mathrm{\Lambda }(t)={\int }_0^t\lambda (s)ds$, and ${\mathrm{\Lambda }}_{{\mathit{Z}}_i}(t)={\int }_0^{t}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)d\mathrm{\Lambda }(s)$. The estimation for MLE of β and the infinite dimensional parameter Λ can be computationally intractable if directly maximizing (10) or solving its score equations. We thereby propose an alternative computational approach, which is a generalization of the EM algorithm for NPMLE discussed in Section 2 under the Cox proportional hazards model.

The semiparametric MLE for the baseline hazard function Λ is obtained by maximizing the likelihood over the set of piece-wise constant functions. Of note, the estimator can have jumps at both censored and uncensored times by observing that the likelihood function achieves its maximum for the hazard function with jumps on {t₁, ···, t_k}, where t₁ < ··· < t_k denotes distinct failure and censored time points. Similar to the argument in (Vardi, 1989, page 754), for any estimator of Λ (t) that jumps outside of the event times {t₁, ···, t_k} one can find a greater likelihood with jumps on {t₁, ···, t_k} only. A detail explanation is in Supplemental Materials available online.

4.2. MLE and EM Algorithm

For i = 1, ···, n, let $T_{ij}^{\ast }$, j = 1, 2, …, m_i be the truncated latent data corresponding to covariate Z_i. We develop the EM algorithm based on the discretized version of Λ(u) =Σ_u≥tj λ_j, where λ_j is the positive jump at time t_j for j = 1, ···, k, and λ = (λ₁, ···, λ_k). For notational convenience, denote f_i(t) = dF (t | Z_i). The log-likelihood based on the complete data is then

$$\sum _{j=1}^k\sum _{i=1}^n\left[I(T_i=t_j)+\sum _{l=1}^{m_i}I(T_{il}^{\ast }=t_j)\right]log{f}_i(t_j)$$

Conditional on the observed data relative to the ith subject, = {X_i, A_i, δ_i, Z_i}, we obtain the expectation that

$$\begin{array}{l}{w}_{ij}=E\left[I(T_i=t_j)+\sum _{l=1}^{m_i}I(T_{il}^{\ast }=t_j)|{\mathcal{O}}_i\right]\\ ={\delta }_{i}I(X_i=t_j)+(1-{\delta }_i)\frac{p_{ij}I(X_i\le t_j)}{{\sum }_{j=1}^k{p}_{ij}I(X_i\le t_j)}+\frac{\widehat{\tau }}{{\mu }_i}(1-t_j/\widehat{\tau })p_{ij}\end{array}$$ (11)

where

$$p_{ij}={\lambda }_{j}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)exp\left\{-\sum _{l=1}^j{\lambda }_{l}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)\right\},\text{and}{\mu }_i=\sum _{j=1}^k{t}_j{p}_{ij}.$$

Thus, the expected complete-data log-likelihood function conditional on the observed data is as follows:

$$\begin{array}{l}{\ell }_E(\mathit{\beta },\mathit{\lambda })=\sum _{i=1}^n\sum _{j=1}^k{w}_{ij}log{f}_i(t_j)\\ =\sum _{j=1}^k{w}_{+j}log{\lambda }_j+\sum _{i=1}^n{w}_{i+}{\mathit{\beta }}^{\prime }{\mathit{Z}}_i-\sum _{l=1}^k\sum _{j=l}^k\sum _{i=1}^n{w}_{ij}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i){\lambda }_l,\end{array}$$

where $w_{+j}={\sum }_{i=1}^n{w}_{ij}$, and $w_{i+}={\sum }_{j=1}^k{w}_{ij}$. In the M-step, we maximize the expected complete-data log-likelihood function conditional on the observed data with respect to the baseline hazard function at t_j, for j = 1, ···, k,

$$\frac{\partial {\ell }_E(\mathit{\beta },\mathit{\lambda })}{\partial {\lambda }_j}=\frac{w_{+j}}{{\lambda }_j}-\sum _{l=j}^k\sum _{i=1}^n{w}_{il}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)=0,$$

which leads to a closed form of λ_j as a function of β, denoted by

$${\lambda }_j(\mathit{\beta })=\frac{w_{+j}}{{\sum }_{l=j}^k{\sum }_{i=1}^n{w}_{il}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)}.$$ (12)

Here, λ_j is the maximizer of the M-step. Next, we maximize the expected complete-data log-likelihood function with respect to β

$$\frac{\partial {\ell }_E(\mathit{\beta },\mathit{\lambda })}{\partial \mathit{\beta }}=\sum _{i=1}^n{w}_{i+}{\mathit{Z}}_i-\sum _{l=1}^k\sum _{j=l}^k\sum _{i=1}^n{w}_{ij}{\mathit{Z}}_{i}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i){\lambda }_l.$$ (13)

By inserting λ_j(β) of (12) into the equation (13), β can be solved from the following equation,

$$\sum _{i=1}^n{w}_{i+}{\mathit{Z}}_i-\sum _{l=1}^k{w}_{+l}\left\{\frac{{\sum }_{i=1}^n{\sum }_{j=l}^k{w}_{ij}{\mathit{Z}}_{i}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)}{{\sum }_{i=1}^n{\sum }_{j=l}^k{w}_{ij}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_i)}\right\}=0,$$ (14)

which is equivalent to maximizing the complete-data profile likelihood function for λ. With the estimated λ_j (j = 1, ···, k) and β, one can update the expectation of the likelihood via w_ij in (11) and repeat the M-step until the estimators of β and λ_j (j = 1, ···, k) converge.

At the M-step, the estimating equation (14) reveals that we may use the existing software for conventional right-censored data to estimate the covariate coefficient β under the Cox proportional hazards model. To simplify the description, consider a model with one covariate Z. First we need to create a vector with a length of nk for the weight function defined by W_nk = (w₁₁, ···, w_1k, w₂₁, ···, w_2k, ···, w_n1, ···, w_nk), which is estimated at the E-step. The corresponding failure time data and covariate vectors are constructed with the same length as W_nk, T_nk = (t₁, ···, t_k, ···, t₁, ···, t_k) and Z_nk = (Z₁, ···, Z₁, ···, Z_n, ···, Z_n), respectively. By using the function “coxph” in S-PLUS (or R) with the “weights” option, we obtain the estimator of β at the M-step from

$$>\text{coxph}(\text{Surv}({\mathit{T}}_{nk},\mathrm{\Delta })\sim {\mathit{Z}}_{nk},\text{weights}={\mathit{W}}_{nk}),$$

where the censoring indicator, Δ = (1, ···, 1), is an identity vector of length nk.

Note that the algorithm computes β and λ iteratively through the EM steps. The value of the complete log-likelihood ℓ_E(β, λ) increases with each EM step. More specifically, our EM algorithm falls in the general scheme of the ECM algorithm, a variation of EM methods proposed by Meng and Rubin (1993). The convergence of our EM algorithm to the local maximizer is guaranteed by the same conditions that ensure the convergence of the ECM algorithm, as proved in Meng and Rubin (1993). The uniqueness of the NPMLE is guaranteed by the Assumption 5 in Appendix A.1.

4.3. Asymptotic Properties

In this section, we establish the strong consistency and asymptotic normality of the MLE under the regularity conditions list in Appendix A.1. For asymptotic proof, we denote the MLE by (β̂_n, Λ̂_n), and let (β₀, Λ₀) be the true value. In Appendix A.4, we prove the strong consistency by the classical Kullback-Leibler information approach, which has been successfully applied for NPMLE in traditional survival analysis (Murphy, 1994; Parner, 1998).

Theorem 1

Under the regularity conditions listed in Appendix A.1, the MLE (β̂_n, Λ̂_n) are consistent: β̂_n converges to β₀, and Λ̂_n(t) converges to Λ₀(t) almost surely and uniformly in t for t ∈ [0, τ] as n → ∞.

The computation for the MLE of Λ is based on the discretized version Λ̂_n(t) = Σ_t≥tj λ̂_j. The existence and uniqueness of the NPMLE can be proved based on the log-likelihood function ℓ_n(β, λ), in terms of {β, λ} where λ ≡ {λ₁, …, λ_k}, and

$${\ell }_n(\mathit{\beta },\mathit{\lambda })=\sum _{i=1}^n\left[{\int }_0^{\tau }{\mathit{\beta }}^{\prime }{\mathit{Z}}_i{dN}_i(t)+\sum _{l=1}^k{\delta }_{i}log{\lambda }_l\right]-\sum _{l=1}^k{\lambda }_l\sum _{i=1}^n{e}^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}1(t_l\le t_i)-\sum _{i=1}^{n}log\left[{\int }_0^{\tau }exp\left(-e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}\sum _{l=1}^k{\lambda }_{l}1(t_l\le s)\right)ds\right].$$ (15)

Let λ̂(·, β) be the maximizer of ℓ_n(β, λ) for given β. The existence and uniqueness of NPMLE are guaranteed by Assumption 5: the information matrix of the profile likelihood evaluated at the true value β₀ is positive defiinite.

Next, we will apply the Z-theorem for the infinite-dimensional estimating equations to prove the weak convergence of the estimators (van der Vaart and Wellner, 1996, Theorem 3.3.1, p. 310). The score equation of β is

$${\mathit{U}}_{1n}(\mathit{\beta },\mathrm{\Lambda })\equiv \frac{1}{n}\sum _{i=1}^n\{{\int }_0^{\tau }{\mathit{Z}}_i{dN}_i(u)-{\int }_0^{\tau }{Y}_i(u){\mathit{Z}}_i{e}^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}d\mathrm{\Lambda }(u)+{\int }_0^{\tau }{\mathit{Z}}_i\left({\int }_u^{\tau }S(v\mid {\mathit{Z}}_i)dv\right)e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}\mathrm{\Lambda }(u)/{\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)\}.$$ (16)

To obtain the MLE of Λ(·), consider a submodel defined by dΛ_η = (1 + ηh)dΛ, where h is a bounded and integrable function. Taking the derivative of ℓ_n(β, Λ_η) with respect to η, evaluating it at η = 0, and setting h(·) = 1(· ≤ t), we have the score equation for Λ

$${\mathit{U}}_{2n}(t,\mathit{\beta },\mathrm{\Lambda })\equiv \frac{1}{n}\sum _{i=1}^n\{{\int }_0^t{dN}_i(u)-{\int }_0^t{Y}_i(u)e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}d\mathrm{\Lambda }(u)+{\int }_0^t\left({\int }_u^{\tau }S(v\mid {\mathit{Z}}_i)dv\right)e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}d\mathrm{\Lambda }(u)/{\mu }_{\mathit{\beta },\mathrm{\Lambda }}({\mathit{Z}}_i)\}.$$ (17)

Denote the vector for score functions by U_n(·, β, Λ) ≡ {U_1n(β, Λ), U_2n(t, β, Λ)}, and its expectation E₀ under the true values (β₀, Λ₀) by

$${\mathit{U}}_0(·,\mathit{\beta },\mathrm{\Lambda })\equiv \{{\mathit{U}}_{10}(\mathit{\beta },\mathrm{\Lambda }),U_{20}(·,\mathit{\beta },\mathrm{\Lambda })\},$$

where U₁₀(β, Λ) = E₀{U_1n(β, Λ)} and U₂₀(t, β, Λ) = E₀{U_2n(t, β, Λ)}. Both the score function U_n and its expectation U₀ are defined on the parameter set × , where the set is assumed to be compact in ℝ^p, and the set consists of nondecreasing functions in the space of functions with bounded variation. Let ψ̂_n = (β̂_n, Λ̂_n), ψ = (β, Λ) and ψ₀ = (β₀, Λ₀).

By the definition of MLE, U_n(·, ψ̂_n) = 0. As the true parameter ψ₀ satisfies U₀(·, ψ₀) = 0, $\sqrt{n}\{{\mathit{U}}_0(·,{\widehat{\mathit{\psi }}}_n)-{\mathit{U}}_0(·,{\mathit{\psi }}_0)\}=\sqrt{n}\{{\mathit{U}}_0(·,{\widehat{\mathit{\psi }}}_n)-{\mathit{U}}_n(·,{\widehat{\mathit{\psi }}}_n)\}$. In the Appendix A.5, we prove a stochastic approximation $\mid \sqrt{n}\{{\mathit{U}}_0(·,{\widehat{\mathit{\psi }}}_n)-{\mathit{U}}_n(·,{\widehat{\mathit{\psi }}}_n)\}-\sqrt{n}\{{\mathit{U}}_n(·,{\widehat{\mathit{\psi }}}_0)-{\mathit{U}}_0(·,{\mathit{\psi }}_0)\}\mid =o_p(1)$. Denote that U̇_ψ0 is the Fréchet derivative of the map U₀(·, ψ) evaluated at ψ₀. By the definition of the Fréchet derivative, ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}\{\sqrt{n}({\widehat{\mathit{\psi }}}_n-{\mathit{\psi }}_0)\}=-\sqrt{n}\{{\mathit{U}}_n(·,{\mathit{\psi }}_0)-{\mathit{U}}_0(·,{\mathit{\psi }}_0)\}+o_P(1)$.

The estimating function evaluated at ψ₀, $\sqrt{n}{\mathit{U}}_n({\mathit{\psi }}_0)=\sqrt{n}\{{\mathit{U}}_n(·,{\mathit{\psi }}_0)-{\mathit{U}}_0(·,{\mathit{\psi }}_0)\}$, is a sum of i.i.d. random quantities. We will prove by the empirical processes theory that $\sqrt{n}{\mathit{U}}_n({\mathit{\psi }}_0)$converges weakly to = ( , ), where is a Gaussian random vector and is a tight Gaussian process. The covariance matrix for is is Σ₁₁ = E₀{U_1n(β₀, Λ₀)^⊗2}, and the covariance between (s) and (t) is Σ₂₂(s, t) = E₀{U_2n(s, β₀, Λ₀)U_2n(t, β₀, Λ₀)}. By the Z-theorem for the infinite-dimensional estimating equations (van der Vaart and Wellner, 1996), we have

Theorem 2

Under the regularity conditions listed in the Appendix A.1, $\sqrt{n}({\widehat{\mathit{\psi }}}_n-{\mathit{\psi }}_0)$converges weakly to a tight mean zero Gaussian process $-{\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}(\mathbb{W})$.

Note that the asymptotic distribution of the sequence $\sqrt{n}({\widehat{\mathit{\psi }}}_n-{\mathit{\psi }}_0)$ is completely determined by the tightness of ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}(\mathbb{W})$ and its marginal covariance function. We characterize the Fréchet derivative U̇_ψ0, viewed as an operator on the parameter space of × . Define for l = 0, 1, 2,

$$\begin{array}{l}{\mathit{K}}_1^{(l)}(u)=E_0\left\{{\mathit{Z}}^{\otimes l}{e}^{{\mathit{\beta }}_0^{\prime }\mathit{Z}}{S}_0(u\mid \mathit{Z}){\mu }_0^{-1}(\mathit{Z}){\int }_0^u{S}_C(s\mid \mathit{Z})ds\right\}\\ {\mathit{K}}_2^{(l)}(t,u)={\int }_u^{\tau }{E}_0\left\{{\mathit{Z}}^{\otimes l}{e}^{2{\mathit{\beta }}_0^{\prime }\mathit{Z}}{\mu }_0^{-1}(\mathit{Z})S_0(v\mid \mathit{Z})\left({\mathrm{\Lambda }}_0(t\wedge v)-{\mathrm{\Phi }}_0(t\mid \mathit{Z}){\mu }_0^{-1}(\mathit{Z})\right)\right\}dv,\end{array}$$

where S_C(u | Z) = P(C ≥ u | Z), μ₀(Z) = μ_{β0, Λ0} (Z), $S_0(t\mid \mathit{Z})=exp\{-exp({\mathit{\beta }}_0^{\prime }\mathit{Z}){\mathrm{\Lambda }}_0(t)\}$, and

$${\mathrm{\Phi }}_0(t\mid \mathit{Z})={\int }_0^t\left({\int }_u^{\tau }{S}_0(v\mid \mathit{Z})dv\right)d{\mathrm{\Lambda }}_0(u).$$

By Assumption 5, the Fisher information of β for known Λ₀ is positive definite

$${\mathit{J}}_0\equiv \left\{{\int }_0^{\tau }{\mathit{K}}_1^{(2)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(2)}(\tau ,u)d{\mathrm{\Lambda }}_0(u)\right\}.$$ (18)

Then the Fréchet derivative U̇_ψ0 can be written in the following form:

$${\stackrel{.}{U}}_{{\mathit{\psi }}_0}(\mathit{\beta },\mathrm{\Lambda })\equiv -\left(\begin{array}{cc}{\mathit{\sigma }}_{11}& {\mathit{\sigma }}_{12}\\ {\mathit{\sigma }}_{\mathbf{21}}& {\sigma }_{22}\end{array}\right)\left(\begin{array}{c}\mathit{\beta }\\ \mathrm{\Lambda }\end{array}\right)\equiv ({\mathit{\sigma }}_{11}(\mathit{\beta })+{\mathit{\sigma }}_{12}(\mathrm{\Lambda }),{\mathit{\sigma }}_{\mathbf{21}}(\mathit{\beta })(·)+{\sigma }_{22}(\mathrm{\Lambda })(·)),$$ (19)

where

$$\begin{array}{l}{\mathit{\sigma }}_{11}(\mathit{\beta })={\left\{{\int }_0^{\tau }{\mathit{K}}_1^{(2)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(2)}(\tau ,u)d{\mathrm{\Lambda }}_0(u)\right\}}^{\prime }\mathit{\beta }\\ {\mathit{\sigma }}_{12}(\mathrm{\Lambda })={\int }_0^{\tau }{\mathit{K}}_1^{(1)}(u)d\mathrm{\Lambda }(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(\tau ,u)d\mathrm{\Lambda }(u)\\ {\mathit{\sigma }}_{\mathbf{21}}(\mathit{\beta })(t)={\left\{{\int }_0^t{\mathit{K}}_1^{(1)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(t,u)d{\mathrm{\Lambda }}_0(u)\right\}}^{\prime }\mathit{\beta }\\ {\sigma }_{22}(\mathrm{\Lambda })(t)={\int }_0^t{K}_1^{(0)}(u)d\mathrm{\Lambda }(u)+{\int }_0^t{K}_2^{(0)}(t,u)d\mathrm{\Lambda }(u).\end{array}$$

We show the invertibility of U̇_ψ0 by translating the operator into the Fredholm integral equations of the second kind (Tricomi, 1985, Chapter 2). We prove in the appendix A.5 that the inverse of ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}$ exists, and the inverse is continuous with the following form

$${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}(\mathit{\beta },\mathrm{\Lambda })\equiv \left(\begin{array}{cc}{\mathit{\sigma }}_{11}^{-1}+{\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}{\mathrm{\Phi }}^{-1}{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}& {\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}{\mathrm{\Phi }}^{-1}\\ {\mathrm{\Phi }}^{-1}{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}& {\mathrm{\Phi }}^{-1}\end{array}\right)\left(\begin{array}{c}\mathit{\beta }\\ \mathrm{\Lambda }\end{array}\right),$$ (20)

where the functional $\mathrm{\Phi }={\sigma }_{22}-{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}$. We shown in the appendix that Φ has an inverse Λ → Φ⁻¹(Λ) expressed in the following form as a function of t

$${\mathrm{\Phi }}^{-1}(\mathrm{\Lambda })(t)={\int }_0^t\frac{d\mathrm{\Lambda }(u)}{K_1^{(0)}(u)}-{\int }_0^{\tau }\left({\int }_0^t\frac{H(u,v)}{K_1^{(0)}(u)}du\right)d\mathrm{\Lambda }(v),$$ (21)

where H(u, v) is the solution of the following integral equation

$$H(u,v)=\frac{Q(u,v)}{K_1^{(0)}(v)}+{\int }_0^{\tau }H(u,s)\frac{Q(s,v)}{K_1^{(0)}(v)}ds,$$ (22)

$$Q(t,u)={\left({\mathit{K}}_1^{(1)}(t){\lambda }_0(t)+{\int }_0^{\tau }{\stackrel{.}{\mathit{K}}}_2^{(1)}(t,v)d{\mathrm{\Lambda }}_0(v)\right)}^{\prime }{\mathit{J}}_0^{-1}\left({\mathit{K}}_1^{(1)}(u)+{\mathit{K}}_2^{(1)}(\tau ,u)\right)-{\stackrel{.}{K}}_2^{(0)}(t,u),$$ (23)

and with the notation ${\stackrel{.}{\mathit{K}}}_2^{(l)}(t,u)=\partial {\mathit{K}}_2^{(l)}(t,u)/\partial t$ for l = 0, 1,

$${\stackrel{.}{\mathit{K}}}_2^{(l)}(t,u)={\int }_u^{\tau }{E}_0\left\{{\mathit{Z}}^{\otimes l}{e}^{2{\mathit{\beta }}_0^{\prime }\mathit{Z}}\frac{S_0(v\mid \mathit{Z})}{{\mu }_0^{-1}(\mathit{Z})}\left({\lambda }_0(t\wedge v)-{\lambda }_0(t){\int }_t^{\tau }\frac{S_0(s\mid \mathit{Z})}{{\mu }_0^{-1}(\mathit{Z})}ds\right)\right\}dv.$$

By Theorem 2, $\sqrt{n}({\widehat{\mathit{\beta }}}_n-{\mathit{\beta }}_0)$ converges in distribution to a mean zero normal random vector characterized by

$${\mathit{\sigma }}_{11}^{-1}({\mathbb{W}}_1)+{\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}{\mathrm{\Phi }}^{-1}{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}({\mathbb{W}}_1)-{\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}{\mathrm{\Phi }}^{-1}({\mathbb{W}}_2),$$ (24)

where the Gaussian process Φ⁻¹( ) has the following form

$${\mathrm{\Phi }}^{-1}({\mathbb{W}}_2)(t)={\int }_0^t\frac{{d\mathbb{W}}_2(u)}{K_1^{(0)}(u)}-{\int }_0^{\tau }\left({\int }_0^t\frac{H(u,v)}{K_1^{(0)}(u)}du\right){d\mathbb{W}}_2(v).$$

Note that the stochastic integral is well defined via the formula of integration by the parts because the functions $u\to 1/K_1^{(0)}(u)$ and $v\to {\int }_0^{t}H(u,v)/K_1^{(0)}(u)du$ are of bounded variation on [0, τ].

Additionally, the process $\sqrt{n}({\widehat{\mathrm{\Lambda }}}_n-{\mathrm{\Lambda }}_0)$ converges weakly to a tight Gaussian process

$${\mathrm{\Phi }}^{-1}{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}({\mathbb{W}}_1)+{\mathrm{\Phi }}^{-1}({\mathbb{W}}_2),$$ (25)

where the processes

$${\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}({\mathbb{W}}_1)(t)={\left\{{\int }_0^t{\mathit{K}}_1^{(1)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(t,u)d{\mathrm{\Lambda }}_0(u)\right\}}^{\prime }{\mathit{J}}_0^{-1}{\mathbb{W}}_1.$$

If the baseline function Λ₀ is known, then $\sqrt{n}({\widehat{\mathit{\beta }}}_n-{\mathit{\beta }}_0)$ converges in distribution to a Gaussian random variate ${\mathit{\sigma }}_{11}^{-1}({\mathbb{W}}_1)$ with mean zero and the sandwich covariance matrix ${\mathit{J}}_0^{-1}{\mathit{\sum }}_{11}{\mathit{J}}_0^{-1}$. Because of the variation associated with the profile-likelihood estimator Λ̂_n, the asymptotic variance of $\sqrt{n}({\widehat{\mathit{\beta }}}_n-{\mathit{\beta }}_0)$ is more complicated, with the extra terms indicated in (24). The variance-covariance matrix may be estimated by its empirical plug-in version, but the computation can be extremely complicated as it needs to solve the integral equation (22). We describe alternative methods for this computation in the following section.

4.4. Variance Estimation

Unlike the estimation of the regression coefficients, variances of the estimator of the MLE, β̂_n cannot be obtained directly from existing software such as R or SAS, because they cannot incorporate the variations caused by the profile likelihood estimator. Instead, we can use bootstrapping techniques or use the information matrix to estimate the variance of the estimators. When working with the observed full likelihood with unknown parameters (β, Λ), the total number of parameters has the same order as the number of observed distinct times, which often yields an information matrix of a high dimension. Murphy and van der Vaart (1999) showed that the inverse of the information matrix for the profile likelihood provides valid variance estimates for the finite-dimensional parameters of interest, i.e., β̂_n under the semiparametric models. There is no general analytical formula for calculating the profile information matrix; thus we describe a numerical EM-aided differentiation approach (Murphy and van der Vaart, 1999; Chen and Little, 1999) to approximate the profile information matrix. Chen and Little (1999) also proved that the score function of the profile likelihood for the observed data is the same as the expected complete-data score function conditional on the observed data at the convergent point. Therefore, the second derivative of the log profile likelihood evaluated at the MLE β̂_n can be approximated by ∂²ℓ_E(β, λ(β))/∂β²|_β=β̂n, which is the first derivative of the expected complete-data score function conditional on the observed data and profiled over λ (β) = (λ ₁(β), ···, λ_k(β)) given by (12). By perturbation around the profile MLE β̂_n, the information matrix for β can then be calculated as follows:

1. Perturb the lth component of β̂_n = (β̂₁, ···, β̂_p) by a small value ε = 1/n in the neighborhood of β̂_l (in both directions). The perturbed estimator is denoted by β̂_∊;,l = (β₁, ···, β_l ± ε, ···, β_k), where l = 1, ···, p.

2. Approximate the lth row of the information matrix of β by

   $$\frac{1}{\epsilon }\left\{{\frac{\partial {\ell }_E(\mathit{\beta },\mathit{\lambda }(\mathit{\beta }))}{\partial \mathit{\beta }}|}_{\mathit{\beta }={\widehat{\mathit{\beta }}}_{\epsilon ,l}}\right\},$$

   where the hazard function λ (β) is obtained from (12) using the M-step described in Section 4.2.

When estimating the variance of λ̂_n, the bootstrap approach can be used. This re-sampling approach is valid, given our established asymptotic normality for the MLE of β̂_n and λ̂_n. In this case, we should obtain the variances of both β̂_n and λ̂_n.

5. SIMULATIONS

We performed simulation studies to evaluate the proposed methods and the corresponding EM algorithms for two settings: nonparametric MLE with an increasing failure rate, and the profile likelihood estimators under the Cox proportional hazards model for length-biased data. We aimed to assess the small sample accuracy and precision of our estimators, and to compare their performance with those of the existing methods under each setting. Each study comprised 1000 repetitions. Sample sizes of 200 and 400 were used.

5.1. Estimating a Distribution Function with an Increasing Failure Rate

We generated independent pairs of (A, T̃) with failure times from a Weibull distribution (F(t) = 1 − exp{−(t/α₂)^α₁}) with α₁ = 2 and α₂ = 1 and truncation times from a uniform distribution to ensure the stationarity assumption. Here, the specified Weibull distribution has an increasing failure rate. The censoring variables measured from the examination time were independently generated from uniform distributions.

Table 1 compares the performance of our proposed estimator and that of Tsai’s estimator (Tsai, 1988), denoted by F_c. When F (t) is greater than 0.5, both estimators achieved outstanding accuracy. In contrast, when F(t) is small, both had downward biases, and the bias decreased with increasing sample sizes. As expected, the empirical standard deviations of our estimator were as much as 25% lower than those of Tsai’s estimator.

Table 1.

Summary Statistics of Simulations for the Estimated Distribution Function with Increasing Failure Rate

Sample Size	C%	F(t)	F̂p(t)			F̂c(t)
			Est.	ESD	ESMSE	Est.	ESD	ESMSE
200	10%	0.10	0.060	0.024	0.047	0.066	0.032	0.047
		0.25	0.211	0.044	0.059	0.211	0.045	0.060
		0.50	0.486	0.045	0.047	0.469	0.046	0.056
		0.75	0.745	0.030	0.030	0.732	0.034	0.038
		0.90	0.898	0.018	0.018	0.892	0.020	0.021
200	30%	0.10	0.068	0.028	0.043	0.065	0.033	0.048
		0.25	0.222	0.045	0.053	0.210	0.046	0.061
		0.50	0.490	0.046	0.047	0.468	0.049	0.058
		0.75	0.748	0.031	0.032	0.732	0.036	0.041
		0.90	0.901	0.019	0.019	0.892	0.022	0.024
400	10%	0.10	0.065	0.019	0.040	0.075	0.025	0.035
		0.25	0.217	0.031	0.045	0.221	0.033	0.043
		0.50	0.495	0.033	0.033	0.478	0.033	0.039
		0.75	0.748	0.021	0.021	0.738	0.023	0.026
		0.90	0.899	0.012	0.012	0.895	0.014	0.015
400	30%	0.10	0.080	0.021	0.029	0.075	0.025	0.036
		0.25	0.236	0.033	0.036	0.221	0.033	0.044
		0.50	0.501	0.033	0.034	0.477	0.034	0.041
		0.75	0.751	0.022	0.022	0.737	0.025	0.028
		0.90	0.901	0.013	0.013	0.895	0.016	0.016

Note: F̂_p(t) is the proposed estimator; F̂_c is Tsai’s conditional estimator; C%=censoring percentage; Est.= average of estimates; ESD = empirical standard deviation; ESMSE = empirical square root of the mean squared error = (bias² + ESD²)^1/2.

5.2. Estimating Regression Coefficients Under the Cox Model

We generated unbiased failure times (T̃) from the proportional hazards model with two covariates, where β = (β₁, β₂) = (0.5, 1), the binary covariate Z₁ ~ Bernoulli(1,0.5), the continuous covariate Z₂ ~ uniform(−0.5,0.5), and the baseline hazard function is t. The censoring times C were independently generated either from uniform distributions or from the specified covariate dependent distributions (see Table 2).

Table 2.

Summary Statistics of Simulations for Estimating Regression Coefficients under Cox Model with β = (β₁, β₂) = (0.5, 1). Mean SE is the mean of the estimated standard errors

Cohort Size	C%	Proposed approach				EE - I		EE - 2		Tsai’s Method
		Est.	ESD	Mean SE	95 % CP	Est.	ESD	Est.	ESD	Est.	ESD
200	15%	(.49,.98)	(.11,.20)	(.11,.19)	(.96,.95)	(.50,1.01)	(.14,.25)	(.51,1.04)	(.13,.24)	(.51,1.01)	(.12,.22)
	30%	(.48,.94)	(.11,.21)	(.11,.20)	(.94,.93)	(.47,.91)	(.19, .33)	(.51,1.01)	(.16,.28)	(.50,1.01)	(.14,.25)
	50%	(.46,.93)	(.12,.21)	(.12,.20)	(.93,.94)	(.42,.85)	(.23,.40)	(.51,1.02)	(.19,.34)	(.49,1.01)	(.18,.31)
400	15%	(.49,.98)	(.08,.14)	(.08,.14)	(.95,.95)	(.50,.99)	(.10,.18)	(.51,1.02)	(.09,.17)	(.52,1.01)	(.09,.15)
	30%	(.48,.97)	(.08,.15)	(.08,.14)	(.93,.93)	(.46,.94)	(.14,.24)	(.50,1.01)	(.11,.21)	(.50,1.00)	(.10,.17)
	50%	(.48,.94)	(.08,.15)	(.08,.15)	(.94,.92)	(.42,.84)	(.18,.31)	(.51,1.02)	(.14,.24)	(.49,1.01)	(.13,.22)
λc = t exp (0.5Z1 + 05Z2).
200	30%	(.48,.95)	(.11,.20)	(.11,.20)	(.94,.93)	(.39,.89)	(.18,.31)	(.46,.97)	(.15,.27)	(.58,1.07)	(.13,.23)
400	30%	(.48,.97)	(.08,.15)	(.08,.14)	(.96,.96)	(.39,.91)	(.13,.23)	(.45,.97)	(.11,.20)	(.58,1.08)	(.10,.17)
λc = t exp (Z2)
200	30%	(.48,.96)	(.11,.20)	(.11,.20)	(.95,.94)	(.52,.82)	(.18,.31)	(.53,.92)	(.15,.28)	(.49,1.15)	(.14,0.24)
400	30%	(.48,.96)	(.08,.15)	(.08,.14)	(.93,.93)	(.50,.80)	(.12,.22)	(.51,.91)	(.11,.19)	(.50,1.15)	(.10,.17)
C ~ Z1U (0, 1) + (1 − Z1)U (0, 1.8)
200	30%	(.49,.96)	(.12,.21)	(.12,.20)	(.95,.93)	(.82,.92)	(.20,.33)	(.71,1.00)	(.17,.28)	(.41,1.02)	(.15,.25)
400	30%	(.48,.96)	(.09,.14)	(.08,.14)	(.92,.95)	(.82,.89)	(.11,.22)	(.69,.97)	(.12,.19)	(.70,.96)	(.09,.17)

For a light (15%) or moderate (30%) censoring percentage, the mean estimates of the coefficients agreed well with the true parameter no matter whether the censoring distribution was dependent on or independent of the covariates. Even with heavy censoring (50%), the inferences associated with the proposed method were fairly accurate for all of the scenarios investigated: the means of the estimated standard errors were close to the empirical standard errors and the coverage of the 95% confidence intervals were reasonable, ranged from 92% to 96%.

In Table 2, we also show a comparison of the performance between the proposed MLE and the existing estimation methods for length-biased data. The estimating equation approaches of Qin and Shen (2010) are as follow:

$$\begin{array}{rr}\hfill EE-I:& \hfill \sum _{i=1}^n{\delta }_i\left[{\mathit{Z}}_i-\frac{{\sum }_{j=1}^{n}I(X_j\ge X_i){\delta }_j{\{X_j{S}_C(X_j-A_j)\}}^{-1}{\mathit{Z}}_{j}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)}{{\sum }_{j=1}^{n}I(X_j\ge X_i){\delta }_j{\{X_j{S}_C(X_j-A_j)\}}^{-1}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)}\right]=0,\\ \hfill EE-II:& \hfill \sum _{i=1}^n{\delta }_i\left[{\mathit{Z}}_i-\frac{{\sum }_{j=1}^{n}I(X_j\ge X_i){\delta }_j{\{W_C(X_j)\}}^{-1}{\mathit{Z}}_{j}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)}{{\sum }_{j=1}^{n}I(X_j\ge X_i){\delta }_j{\{W_C(X_j)\}}^{-1}exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)}\right]=0,\end{array}$$

where S_C(·) is the survival function of the censoring time C, and $W_C(t)={\int }_0^t{S}_C(s)ds$. Tsai (2009) proposed the pseudo-partial likelihood method with the following estimating equations based on the score statistics,

$$\begin{array}{rr}\hfill EE-PL:& \hfill \sum _{i=1}^n{\int }_0^{\infty }\left\{{\mathit{Z}}_i-\frac{{\sum }_{j=1}^n{\mathit{Z}}_j{Y}_j(t)exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)W(t,X_j)}{{\sum }_{j=1}^n{Y}_j(t)exp({\mathit{\beta }}^{\prime }{\mathit{Z}}_j)W(t,X_j)}\right\}{dN}_i(t)=0,\end{array}$$

where $W(t,X_j)={\delta }_j\frac{W_C(X_j)-W_C(X_j-t)}{W_C(X_j)}+(1-{\delta }_j)\frac{S_C(X_j)-S_C(X_j-t)}{S_C(X_j)}$.

When the censoring was independent of the covariates, all the types of estimators had a small bias with light censoring (15%). With moderate (30%) or heavy censoring (50%), the biases associated with the MLE were much smaller than those associated with EE I. The MLE method always exhibited clearly superior efficiency, with smaller empirical standard errors. For instance, the standard errors associated with the estimating equations were 1.12 to 1.62 times greater, and the standard errors associated with the pseudo-partial likelihood method were 1.09 to 1.50 times greater than those associated with the MLE method based on a sample size of 200. When the censoring distribution was dependent on the covariates, the estimators obtained by EE-I, EE-II and EE-PL were biased compared with those obtained by the MLE method. In summary, the MLE approach is the most efficient one among the four methods, and is also the most robust to various censoring mechanisms.

6. A REAL DATA EXAMPLE

Dementia is a progressive degenerative medical condition and is one of the leading causes of all deaths in the United States and Canada. The Canadian Study of Health and Aging was a multicenter epidemiologic study of dementia, in which 14,026 subjects 65 years or older were randomly chosen throughout Canada to receive an invitation for a health survey. A total of 10,263 subjects agreed to participate in the study (Wolfson et al., 2001). The participants were then screened for dementia, and 1132 people were identified as having the disease. The individuals with dementia were followed until their deaths or last follow-up dates in 1996, and their dates of dementia onset were ascertained from their medical records.

After excluding subjects with missing data regarding the date of disease onset or classification of dementia subtype, a total of 818 patients remained {393 with probable Alzheimer’s disease, 252 with possible Alzheimer’s disease and 173 with vascular dementia. Other study variables included the approximate date of dementia onset, date of screening for dementia, date of death or censoring and death indicator variable. Given the prevalent cases ascertained cross-sectionally, Asgharian et al. (2006) validated the stationarity assumption, which was defined as that the incidence of dementia did not change over the period of the study.

At the end of the study, 638 out of 818 patients had died and the others were right censored. Within this elderly cohort, it seems reasonable to assume that the overall death rate increases with age. Applying the NPMLE approaches described in Sections 2 and 3, we estimated the hazard function for each subtype of dementia, and plotted the survival of patients with probable Alzheimer’s disease, possible Alzheimer’s disease and vascular dementia with and without the constraint of the increasing risk of death (see Figure 1). It is not surprising that the estimated survival curves with the constraint, i.e., additional information, have narrower confidence intervals than their corresponding survival curves without the constraint. As pointed out by one referee, a monotone hazards constraint may not hold for death due to dementia, particularly for patients with vascular dementia, because of the uncertainty associated with the cause of death.

Figure 1.

Estimated survival curves according to subtypes of dementia, with and without the constraint of increasing risk of death with age

Using the diagnosis subtype of possible Alzheimer’s disease as the baseline cohort, we defined two indicator variables for the other two subtypes of dementia for the Cox proportional hazards model. Applying the proposed method in Section 4, the estimated covariate Effects of two subtypes of dementia and their standard errors are listed in Table 3. The results showed that the long-term survival distributions were statistically significantly different between the group with vascular dementia and the group with possible Alzheimer’s dementia, and marginally different between the group with probable Alzheimer’s dementia and the group with possible Alzheimer’s dementia. We also analyzed the same data set using the estimating equation methods (EE-I and EE-II) by Qin and Shen (2010) and the pseudo-partial likelihood method by Tsai (2009). The estimated coefficients with the associated standard errors obtained by both EE-II and EE-PL indicated no statistically significant survival differences between the three subtypes of dementia. The results from EE-I suggested there existed a statistically significant survival difference between the group with vascular dementia and the group with possible Alzheimer’s dementia, but no statistically significant survival differences between the group with vascular dementia and the group with probable Alzheimer’s dementia. The discrepancy in the inferences is most likely caused by the loss in efficiency when using the estimating equation method or the pseudo-partial likelihood method compared to using the MLE method.

Table 3.

Estimates (Standard Errors) of Regression Coefficients Using Length-biased Adjusted Methods for Dementia Data.

	MLE	EE-I	EE-II	EE-PL
Probable Alzheimer	0.125 (0.062)	0.109 (0.092)	0.134 (0.091)	0.064(0.081)
Vascular Dementia	0.185 (0.077)	0.245 (0.110)	0.208 (0.110)	0.164(0.111)

7. CONCLUDING REMARKS

We have proposed new EM algorithms for length-biased data to obtain full likelihood maximum estimators under three settings, and the missing data mechanism in the EM-algorithm is the left truncation for the length-biased data. In constrast to Vardi’s (1989) EM algorithm for estimating nonparametric survival distributions, the advantage of the new EM algorithm is that one can directly estimate the non-parametric survival distribution or hazard function of the unbiased failure time T̃.

One major challenge to maximum likelihood estimation when it involves infinite dimensional parameters is computational intractability. We have implemented the new EM algorithm together with the profile likelihood method for jointly estimating the baseline hazard function and the covariate coefficients under the Cox regression model for length-biased data. Commercially available statistics software for the Cox model can be adapted for easy computation. The EM algorithm is not computational intensive even with continuous covariates, since p_ij and λ_j can be obtained easily from the closed form expressions.

Similar to the NPMLE for traditional survival data, the method we have proposed requires the observation of at least one failure time to ensure that the large sample properties hold in the settings for length-biased data. As shown in our empirical studies, the proposed computational algorithms to solve MLE perform well in terms of accuracy, and are more efficient compared to the existing estimating equation approaches, which are more efficient than the conditional approach (Wang et al., 1993). Without assuming a known parametric distribution of Z as in Bergeron et al. (2008), maximizing the likelihood function (1) is equally efficient to maximinizing the full likelihood including the marginal distribution of Z.

Parallel to the observation of Zeng and Lin (2007) for traditional survival data, estimators obtained from MLE are much more robust and efficient than those from the estimating equation approaches, and well suited for the proportional hazards regression model and other nonparametric estimations for length-biased right-censored data. The proposed EM algorithm may be further generalized to other semiparametric models, and the tools for model checking should be developed for length-biased right-censored data.

A APPENDIX

A.1 Assumptions

Denote the Euclidean norm by |·|. For a vector z = (z₁, …, z_p)′, |z| ≡ (|z₁|² + ··· + |z_p|²)^1/2. To avoid measurability issues, the probability is understood as the outer probability (van der Vaart and Wellner, 1996). We adopt the convention that 0/0 = 0. We assume the following regularity assumptions:

1. The true value of the hazard function λ₀(·) is continuously differentiable. In addition, the upper bound τ of the support of the cumulative hazard function Λ₀ is finite.

2. The parameter β is in a compact set that contains β₀. The parameter set for the baseline function contains all nondecreasing functions Λ satisfying Λ (0) = 0 and Λ (τ) < ∞.

3. The residual censoring time C satisfies P(C > V) > 0. Its survival function S_C(·) is continuous.

4. For the covariate vector Z, the terms E₀|Z|² and E₀|e^β′Z| are bounded.

5. The information matrix −∂²E[ℓ_n(β, λ̂ (·, β)]/∂β² evaluated at the true values β₀ is positive definite.

6. If P(b′Z = c₀) = 1 for some constant c₀, then b = 0.

Assumption 1 and 3 imply that the bivariate function Q(t, u) defined in (23) is continuous on [0, τ] × [0, τ]. Assumption 3 implies that the censoring may occur after V. Assumption 5 is a classical condition that appears in the study of the Cox model for traditional survival data (Andersen et al., 1992, Condition VII2.1.(e), page 497). It implies that the matrix J₀ defined in (18) is positive definite, which is the information matrix for β when the baseline function Λ₀ is known. The positive definiteness of J₀ can be also implied by the fact the model is identifiable (Rothenberg, 1971). Assumption 6 means that there is no covariate colinearity, which ensure the model identifiability.

A.2 Proof of the lemma on the convergence of τ̂ = t_k to τ

Recall that X_i = min(A_i+V_i, A_i+C_i), and t_k = max{X₁, ···, X_n}. For any arbitrary small ε > 0,

$$\begin{array}{l}P(\mid \underset{1\le i\le n}{max}{X}_i-\tau \mid >\epsilon )={[P(X_1<\tau -\epsilon )]}^n={\{1-E[I(A_1+V_1>\tau -\epsilon )S_C(\tau -\epsilon -A_1)]\}}^n\\ ={\left\{1-{\int }_{\tau -\epsilon }^{\tau }w(y)f(y)dy/\mu \right\}}^n\\ ={\{1-w(\xi )f(\xi )\epsilon /\mu \}}^n,\end{array}$$

where $w(y)={\int }_0^y{S}_C(\tau -\epsilon -a)da>0,\tau -\epsilon \le \xi \le \tau ,f(\xi )>0$, and the last equality uses the mean value theorem. Given the above equation, for any arbitrary small η > 0,

$$P(n^{\eta }\mid \underset{1\le i\le n}{max}{X}_i-\tau \mid >\epsilon )={\{1-w(\xi )f(\xi )\epsilon n^{-\eta }/\mu \}}^n,$$

where τ − εn^−η ≤ ξ ≤ τ. Note that when n → ∞,

$$\begin{array}{c}w(\xi )={\int }_0^{\xi }{S}_C(\tau -\epsilon n^{-\eta }-a)da={\int }_{\tau -\xi -\epsilon n^{-\eta }}^{\tau -\epsilon n^{-\eta }}{S}_C(s)ds\to {\int }_0^{\tau }{S}_C(s)ds,\\ {\int }_0^{\tau }{S}_C(s)ds=E[{\int }_0^{\tau }I(C>s)ds]=E[min(C,\tau )].\end{array}$$

Thus, as long as E[C] > 0 which is inferred by Assumption 3, we have w(ξ) → w(τ) > 0 and f(ξ) → f(τ) > 0, for n → ∞, since τ is the upper bound for the support of population time and Λ(τ) < ∞. Therefore, when n → ∞ we have that

$$P(n^{\eta }\mid \widehat{\tau }-\tau \mid >\epsilon )\approx exp(-w(\tau )f(\tau )\epsilon n^{1-\eta }/\mu ),$$

which implies that τ̂ → τ in a rate higher than n^1/2 but lower than n (i.e. when 1/2 < η < 1, the above probability converges to zero).

Note that by the Borel-Cantelli Lemma, τ̂_n → τ in probability implies that for every subsequence, there is a sub-subsequence {n′} such that τ̂_n′ → τ almost surely (Ferguson, 1996, page 8). This combined with the fact that τ can be consistently estimated by τ̂_n complete the proof for strong consistency of Λ̂_n. As τ̂_n ≤ τ, these facts also infer that the mean μ of the population failure time T̃ can be consistently estimated by ${\int }_0^{\widehat{\tau }}exp(-{\widehat{\mathrm{\Lambda }}}_n(s))ds$.

A.3 Consistency of Λ̂_n with increasing failure rate

Let ||·||_τ denote the supremum norm over [0, τ]. We have the following strong consistency result for the NPMLE Λ̂_n(·) proposed in Section 3:

Proposition 1

Suppose Λ is convex on its support [0, τ]. Under the regularity conditions,

$${\left|\right|{\widehat{\mathrm{\Lambda }}}_n-\mathrm{\Lambda }\left|\right|}_{\tau }\le {\left|\right|{\mathrm{\Lambda }}_n-\mathrm{\Lambda }\left|\right|}_{\tau }{\to }_{a.s.}0.$$

We adapt the proof in Huang and Wellner (1995). Let ε_n = ||Λ_n − Λ||_τ. As argued in Asgharian and Wolfson (2005), ε_n → 0 almost surely when n → ∞. Since Λ is convex on [0, τ], it must be continuous on [0, τ]. The function Λ − ε_n is convex and is a minorant of Λ_n, i.e., Λ (s) − ε_n ≤ Λ_n(s) for all 0 ≤ s ≤ τ. By the definition of Λ̂_n, we have that for all 0 ≤ s ≤ τ,

$$\mathrm{\Lambda }(s)-{\epsilon }_n\le {\widehat{\mathrm{\Lambda }}}_n(s)\le {\mathrm{\Lambda }}_n(s).$$

It follows that −ε_n ≤ Λ̂_n(s) − Λ(s) ≤ Λ_n(s) − Λ(s) ≤ ε_n for all 0 ≤ s ≤ τ. The conclusion of the proposition follows as ||Λ̂_n − Λ||_τ ≤ ε_n goes to 0 almost surely.

A.4 Consistency: Proof of Theorem 1

Note that the log-likelihood function ℓ_n(β, λ) is strictly concave in λ as each function of λ in ℓ_n(β, λ) is concave or strictly concave and the summation of concave functions is concave. Hence, for each β in a compact set , we can find a unique maximizer of λ̂(·, β) of the likelihood function l_n(β, λ). The existence of the NPMLE for {β, λ} follows by the compactness of for the profile likelihood ℓ_n(β, λ̂(·, β)), which is continuous in β. The uniqueness of the NPMLE is guaranteed by Assumption 5 for large samples.

The technical details of the consistency proof are similar to those of Murphy (1995) or Parner (1998). We provide only a sketch of the proof. As the MLE (β̂_n, Λ̂_n) maximizes the log-likelihood function ℓ_n, the method is to use the empirical Kullback-Leibler distance ℓ_n(β̂_n, Λ̂_n) − ℓ_n(β₀, Λ₀), which must always be nonnegative. If (β̂_n, Λ̂_n) converges at all, say, to (β*, Λ*), then ℓ_n(β̂_n, Λ̂_n) − ℓ_n(β₀, Λ₀) must converge to the negative Kullback-Leibler distance between P_{β*; Λ*} and P_{β0, Λ0} by the strong law of large numbers, where P_β; _Λ is the probability measure under the parameter (β, Λ). The Kullback-Leibler distance between P_{β*; Λ*} and P_{β0, Λ0} therefore must be zero, and we conclude that P_{β*; Λ*} = P_{β0, Λ0} almost surely. It then follows by model identifiability that β* = β₀ and Λ* = Λ₀.

We need to find, for any subsequence of (β̂_n, Λ̂_n), a further convergent subsequence. The first step is to show that (β̂_n, Λ̂_n) stays bounded. As β̂_n is in a compact set, it must stay bounded. Because (β̂_n, Λ̂_n) maximizes the likelihood function, ℓ_n(β̂_n, Λ̂_n) − ℓ_n(β̄, Λ̄) ≥ 0 for each (β̄, Λ̄) in the parameter set. Recall that τ̂ = t_k. We show that Λ̂_n(τ̂) stays bounded, ${\overline{lim}}_n{\widehat{\mathrm{\Lambda }}}_n<\infty$. We use the method of contradiction. Suppose that Λ̂_n(τ̂) diverges. Then we can construct some sequence {β̄_n, Λ̄_n} such that the empirical Kullback-Leibler distance ℓ_n(β̂_n, Λ̂_n) − ℓ_n(β̄_n, Λ̄_n) would become negative infinity. This is a contradiction as the Kullback-Leibler distance is always nonnegative. The construction of the contradiction is along the lines as in Murphy (1994). Briefly, we choose β̂_n = β₀ and define Λ̄_n to be

$${\overline{\mathrm{\Lambda }}}_n(t)={\int }_0^t{\left\{\sum _{i=1}^n{W}_i(u,{\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)\right\}}^{-1}d\left\{\sum _{j=1}^n{N}_j(u)\right\},$$ (26)

where $W_i(u,\mathit{\beta },\mathrm{\Lambda })=e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}(Y_i(u)-{\int }_u^{\tau }S(v\mid {\mathit{Z}}_i)dv/{\int }_0^{\tau }S(v\mid {\mathit{Z}}_i)dv)$, and note that

$${EW}_i(u,\mathit{\beta },\mathrm{\Lambda })-E\left[e^{{\mathit{\beta }}^{\prime }{\mathit{Z}}_i}\{u-E_C{(u-C)}^{+}\}S(u\mid \mathit{Z})/\mu (\mathit{Z})\right]\ge 0.$$

It can be shown easily that Λ̄_n converges to Λ₀ almost surely and uniformly in t. By a technical argument similar to that of Murphy (1995), we can show that ℓ_n(β̂_n, Λ̂_n) − ℓ_n(β̄_n, Λ̄_n) → −∞ as n → ∞. This is impossible so Λ̂_n must stay bounded.

As Λ̂_n stays bounded, we can apply Helly’s selection principle to find a convergent subsequence of (β̂_nk Λ̂_nk) for an arbitrary subsequence from the sequence indexed; by {1, ···, n}. By the strong law of large numbers, such convergent subsequence must converge to (β₀, Λ₀) using the classical Kullback-Leibler information approach. For any given subsequence {n_k}, we can identify a further subsequence of (β̂ _nk, Λ̂ _nk) that converges to (β₀, Λ₀). Helly’s selection theorem implies that the entire sequence (β̂_n, Λ̂_n(t)) must converge to (β₀, Λ₀(t)) for each t ∈ [0, τ]. By the assumption that Λ₀(·) is monotone and continuous, the convergence of Λ̂_n(t) ∈ [0, τ] at each t is also uniformly in t. The convergence is also almost surely a true convergence as the proof is carried out for a fixed ω in the underlying probability space Ω, where we use the law of large numbers, except that it is countable many times.

A.5 Asymptotic Normality: Proof of Theorem 2

We prove the asymptotic normality by the Z-theorem for the infinite-dimensional estimating equations (van der Vaart and Wellner, 1996, Theorem 3.3.1, page 310). This approach has been successfully applied by (Murphy, 1995, Theorem 1) and (Parner, 1998, Theorem 2) among many others. The proof requires the confirmation of the three main conditions of the Z-theorem: Fréchet differentiability, weak convergence of $\sqrt{n}{\mathit{U}}_n({\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)$ and a stochastic approximation of the estimating equations, which we outline below.

Fréchet Derivative and its Invertibility

Let ${\mathrm{\Lambda }}_{\mathit{Z}}(t)={\int }_0^t{e}^{{\beta }^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)$. We first show that the population estimating equation U₀ is Fréchet differentiable and its Fréchet derivative is continuously invertible. We have defined U₀(·, β, Λ) = (U₁₀(β, Λ), U₂₀(·, β, Λ)) where

$$\begin{array}{l}{\mathit{U}}_{10}(\mathit{\beta },\mathrm{\Lambda })={\int }_0^{\tau }{E}_0\left\{\mathit{Z}dN(t)-Y(u)\mathit{Z}d{\mathrm{\Lambda }}_{\mathit{Z}}(u)+\mathit{Z}\left({\int }_u^{\tau }S(v\mid \mathit{Z})dv\right)d{\mathrm{\Lambda }}_{\mathit{Z}}(u)/{\mu }_{\beta ,\mathrm{\Lambda }}(\mathit{Z})\right\},\\ U_{20}(t,\mathit{\beta },\mathrm{\Lambda })={\int }_0^t{E}_0\left\{dN(u)-Y(u)d{\mathrm{\Lambda }}_{\mathit{Z}}(u)+\left({\int }_u^{\tau }S(v\mid \mathit{Z})dv\right)d{\mathrm{\Lambda }}_{\mathit{Z}}(u)/{\mu }_{\beta ,\mathrm{\Lambda }}(\mathit{Z})\right\}.\end{array}$$

The Fréchet derivative can be calculated from the Gâteaux variations of U₀(β, Λ) at (β₀, Λ₀). That is done to differentiate U₀(β_η, Λ_η) with respect to η and evaluated at η = 0, where the submodels are β_η = β₀ + ηβ and Λ_η = Λ₀ + ηΛ.

The Gâteaux derivative of U₂₀(t, β, Λ) evaluated at (β₀, Λ₀) is

$$-\{{\mathit{\sigma }}_{\mathbf{21}}(\mathit{\beta })+{\sigma }_{22}(\mathrm{\Lambda })\},$$

where ${\mathit{\sigma }}_{\mathbf{21}}(\mathit{\beta })\equiv \partial /\partial \eta U_{20}(t,{\mathit{\beta }}_{\eta },{\mathrm{\Lambda }}_0){\mid }_{\eta =0}={\mathit{\beta }}^{\prime }\{{\int }_0^t{\mathit{K}}_1^{(1)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(t,u)d{\mathrm{\Lambda }}_0(u)\}$, and ${\sigma }_{22}(\mathrm{\Lambda })\equiv \partial /\partial \eta U_{20}(t,{\mathit{\beta }}_0,{\mathrm{\Lambda }}_n){\mid }_{\eta =0}={\int }_0^t{K}_1^{(0)}(u)d\mathrm{\Lambda }(u)+{\int }_0^{\tau }{K}_2^{(0)}(t,u)d\mathrm{\Lambda }(u)$. The Gâteaux derivative of U₁₀(t, β, Λ) evaluated at (β ₀, Λ₀) is

$$-\{{\mathit{\sigma }}_{11}(\mathit{\beta })+{\mathit{\sigma }}_{12}(\mathrm{\Lambda })\},$$

where ${\mathit{\sigma }}_{11}(\mathit{\beta })\equiv \partial /\partial \eta {\mathit{U}}_{10}({\mathit{\beta }}_{\eta },{\mathrm{\Lambda }}_0){\mid }_{\beta ={\mathit{\beta }}_0}={\{{\int }_0^{\tau }{\mathit{K}}_1^{(2)}(u)d{\mathrm{\Lambda }}_0(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(2)}(\tau ,u)d{\mathrm{\Lambda }}_0(u)\}}^{\prime }\mathit{\beta }$, and ${\mathit{\sigma }}_{12}(\mathrm{\Lambda })\equiv \partial /\partial \eta {\mathit{U}}_{10}({\mathit{\beta }}_0,{\mathrm{\Lambda }}_{\eta }){\mid }_{\eta =0}={\int }_0^{\tau }{\mathit{K}}_1^{(1)}(u)d\mathrm{\Lambda }(u)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(\tau ,u)d\mathrm{\Lambda }(u)$.

To obtain the results on weak convergence, we need to strengthen the Ĝateaux differentiability to Fréchet differentiability, essentially for the proof of tightness (van der Vaart and Wellner, 1996, page 310). The Fréchet differentiability of U₀(β, Λ) can be confirmed by definition; its derivative has the form in (19). Note that the operator U̇_ψ0 is a linear continuous operator defined on parameter space in the product space of ℝ^p and the Banach space L₂[0, τ]. If the inverse operator ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}$ exists, then it must be continuous by Banach’s continuous inverse theorem (Zeidler, 1995, page 179). Hence, to prove the continuous invertibility of U̇_ψ0, we only need to show the existence of the inverse operator ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}$.

To show the existence of the inverse operator ${\stackrel{.}{U}}_{{\mathit{\psi }}_0}^{-1}$, we only need to show by the formula in (20) that σ₁₁ and $\mathrm{\Phi }={\sigma }_{22}-{\mathit{\sigma }}_{\mathbf{21}}{\mathit{\sigma }}_{11}^{-1}{\mathit{\sigma }}_{12}$ have inverse. The operator ${\mathit{\sigma }}_{11}(\mathit{\beta })={\mathit{J}}_0^{\prime }\mathit{\beta }$ is a linear operator, where the matrix J₀ defined in (18) is the Fisher information for β for known Λ₀. By Assumption 5, the matrix J₀ has an inverse, and hence σ₁₁ is invertible. The operator Φ has the following form:

$$\mathrm{\Phi }(\mathrm{\Lambda })=-\left[{\int }_0^t{K}_1^{(0)}(u)d\mathrm{\Lambda }(u)+{\int }_0^{\tau }\left\{K_2^{(0)}(t,u)-{\left({\int }_0^t{\mathit{K}}_1^{(1)}(v)d{\mathrm{\Lambda }}_0(v)+{\int }_0^{\tau }{\mathit{K}}_2^{(1)}(t,v)d{\mathrm{\Lambda }}_0(v)\right)}^{\prime }{\mathit{J}}_0^{-1}·\left({\mathit{K}}_1^{(1)}(u)+{\mathit{K}}_2^{(1)}(\tau ,u)\right)\right\}d\mathrm{\Lambda }(u)\right].$$

The invertibility of Φ is equivalent to show that there exits a unique solution to the equation Φ(Λ) = Λ̃ for a function Λ̃ of bounded variation. Taking the derivative with respect to t on both sides of the equation,

$$-d\stackrel{\sim }{\mathrm{\Lambda }}(t)=K_1^{(0)}(t)d\mathrm{\Lambda }(t)-{\int }_0^{\tau }Q(t,u)d\mathrm{\Lambda }(u),$$ (27)

where Q(t, u) is defined previously in (23). We observe that the integral equation (27) is the Fredholm equation of the second type. By Assumptions 1 and 3, the bivariate function Q(t, u) defined in (23) is continuous on [0, τ] × [0, τ]. Also by assumption 3, the function $1/K_1^{(0)}(t)$ is continuous and bounded away from 0 for t > 0. By the classical theory for integral equation (Tricomi, 1985, Chapter 2), there is a unique solution dΛ (t) to the Fredholm integral equation (27), characterized by

$$d\mathrm{\Lambda }(t)=-\frac{d\stackrel{\sim }{\mathrm{\Lambda }}(t)}{K_1^{(0)}(t)}+{\int }_0^{\tau }\frac{H(t,u)}{K_1^{(0)}(t)}d\stackrel{\sim }{\mathrm{\Lambda }}(u),$$

where H(t, u) satisfies the equation (22). Finally, the invertibility of the functional Φ follows and its inverse operator Φ⁻¹(Λ) has a form expressed in (21).

Weak Convergence of $\sqrt{n}{\mathit{U}}_n({\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)$

As the true value (β₀, Λ₀) satisfies U₀(β₀, Λ₀) = 0, we have

$$\sqrt{n}{\mathit{U}}_n({\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)=\sqrt{n}\left\{{\mathit{U}}_{1n}({\mathit{\psi }}_0)-{\mathit{U}}_{10}({\mathit{\psi }}_0),U_{2n}(t,{\mathit{\psi }}_0)-U_{20}(t,{\mathit{\psi }}_0)\right\}.$$

By the multivariate central limit theorem for the summation of independently and identically distributed (i.i.d.) random vectors, $\sqrt{n}\{{\mathit{U}}_{1n}({\mathit{\psi }}_0)-{\mathit{U}}_{10}({\mathit{\psi }}_0)\}$converges in law to , provided that the second moment is finite. The process $\sqrt{n}\{U_{2n}(t,{\mathit{\psi }}_0)-U_{20}(t,{\mathit{\psi }}_0)\}$ is a sum of i.i.d. processes of bounded variation on [0, τ]. By a lemma for the central limit theorem for processes of bounded variation (van der Vaart and Wellner, 1996, Example 2.11.16), $\sqrt{n}\{U_{2n}(t,{\mathit{\psi }}_0)-U_{20}(t,{\mathit{\psi }}_0)\}$ converges to a tight Gaussian process, say , provided that the second moment is finite. The weak convergence of $\sqrt{n}{\mathit{U}}_n({\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)$ follows by the continuous mapping theorem.

Stochastic Approximation

To apply the Z-theorem for the infinite dimensional estimating equations (van der Vaart and Wellner, 1996, Theorem 3.3.1), we need to confirm the following stochastic approximation:

$$\sqrt{n}\{({\mathit{U}}_n-{\mathit{U}}_0)({\widehat{\mathit{\psi }}}_n)-({\mathit{U}}_n-{\mathit{U}}_0)({\mathit{\psi }}_0)\}\equiv \sqrt{n}[\{{\mathit{U}}_n(·,{\widehat{\mathit{\psi }}}_n)-{\mathit{U}}_0(·,{\widehat{\mathit{\psi }}}_n)\}-\{{\mathit{U}}_n(·,{\mathit{\psi }}_0)-{\mathit{U}}_0(·,{\mathit{\psi }}_0)\}]=o_P(1).$$

The function is defined on × , where by Assumption 2, the set is compact that contains β₀, and is a set of nondecreasing function such that for each Λ ∈ , Λ (0) = 0 and Λ (τ) < ∞. Hence contains Λ₀. To apply the Z-theorem, we need Λ in the closed linear subspace generated by the set . The subspace is viewed in the space of functions of bounded variation on [0, τ] endowed with the variation norm by || · ||_v, defined by the total variation of Λ on [0, τ], i.e.,

$${\left|\right|\mathrm{\Lambda }\left|\right|}_v\equiv sup\sum _{k=1}^{m^{\prime }}\mid \mathrm{\Lambda }(s_k)-\mathrm{\Lambda }(s_{k-1})\mid ,$$

where the supremum is taken over all finite partitions of [0, τ], {0 = s₀ < s₁ < ··· < s_m′ = τ}.

We first derive the score functions for β and Λ (·) for a single observation . We keep in the nations for the score functions to emphasize its dependence on the data. By straightforward calculation, the score function for β is

$${\stackrel{.}{\mathit{\ell }}}_1(\mathcal{O},\mathit{\beta },\mathrm{\Lambda })={\int }_0^{\tau }\mathit{Z}dN(u)-{\int }_0^{\tau }Y(u){\mathit{Z}e}^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)+{\int }_0^{\tau }\mathit{Z}\left({\int }_u^{\tau }S(v\mid \mathit{Z})dv\right)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)/{\mu }_{\mathit{\beta },\mathrm{\Lambda }}(\mathit{Z}).$$ (28)

For the infinite dimensional parameter Λ (·), consider a submodel defined by dΛ _η = (1 + ηh)dΛ, where h is a bounded and integrable function. By taking the derivative of ℓ_n(β, Λ_η) with respect to η and evaluating it at η = 0, we have the score operator for Λ

$${\stackrel{.}{\ell }}_2(\mathcal{O},\mathit{\beta },\mathrm{\Lambda })(h)={\int }_0^{\tau }h(v)dN(v)-{\int }_0^{\tau }Y(v)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}h(v)d\mathrm{\Lambda }(v)+{\int }_0^{\tau }\left\{{\int }_u^{\tau }S(v\mid \mathit{Z})dv\right\}h(u)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)/{\mu }_{\mathit{\beta },\mathrm{\Lambda }}(\mathit{Z}).$$ (29)

Taking h(·) = 1(· ≤ t) in (29), we have the score function for Λ

$${\stackrel{.}{\ell }}_2(t;\mathcal{O},\mathit{\beta },\mathrm{\Lambda })={\int }_0^{t}dN(v)-{\int }_0^{t}Y(v)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}h(v)d\mathrm{\Lambda }(v)+{\int }_0^t\left\{{\int }_u^{t}S(v\mid \mathit{Z})dv\right\}{e}^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)/{\mu }_{\mathit{\beta },\mathrm{\Lambda }}(\mathit{Z}).$$ (30)

We can write ${\stackrel{.}{\mathit{\ell }}}_1(\mathcal{O},\mathit{\beta },\mathrm{\Lambda })={\int }_0^{\tau }\mathit{Z}d{\stackrel{.}{\ell }}_2(t;\mathcal{O},\mathit{\beta },\mathrm{\Lambda })$.

Let ℓ̇(t, , ψ) = {ℓ̇₁ ( , ψ), ℓ̇₂(t, , ψ)}, where ψ = (β, Λ). We first introduce some notations from the empirical processes theory. Let ℙ_n be the empirical probability measure. Then

$${\mathit{U}}_n(\mathit{\beta },\mathrm{\Lambda })={\mathbb{P}}_n\stackrel{.}{\ell }(·,\mathcal{O},\mathit{\psi })=n^{-1}\sum _{i=1}^n\stackrel{.}{\ell }(·,{\mathcal{O}}_i,\mathit{\psi }).$$

Denote the empirical process by ${\mathbb{G}}_{n}f=\sqrt{n}({\mathbb{P}}_{n}f-P_{0}f)$, where P₀ is the expectation under ψ₀. Note that $\sqrt{n}({\mathit{U}}_n-{\mathit{U}}_0)(\mathit{\psi })={\mathbb{G}}_n\stackrel{.}{\mathit{\ell }}(·,\mathcal{O},\mathit{\psi })$ is the empirical process indexed by the class of functions

$$\{\stackrel{.}{\mathit{\ell }}(t;\mathcal{O},\mathit{\psi }),\mathit{\psi }\in \mathcal{B}\times ,\overline{\mathcal{A}},t\in [0,\tau ]\}.$$

Let the norm ||·|| on = × be defined as ||(β, Λ)|| = |β| + ||Λ||_v. Then the stochastic condition that we want to confirm is

$${\left|\right|{\mathbb{G}}_n\stackrel{.}{\ell }(t,\mathcal{O},{\widehat{\mathit{\psi }}}_n)-{\mathbb{G}}_n\stackrel{.}{\ell }(t,\mathcal{O},{\mathit{\psi }}_0)\left|\right|}_{\mathcal{H}}=o_P(1).$$

To apply the functional central limit theory, we show that the class of functions {ℓ̇(t, , ψ) − ℓ̇(t, , ψ₀): ||ψ|| <, δ, t ∈ [0, τ]} is P₀-Donsker. As ${\stackrel{.}{\mathit{\ell }}}_1(\mathcal{O},\mathit{\psi })={\int }_0^{\tau }\mathit{Z}d{\stackrel{.}{\ell }}_2(t,\mathcal{O},\mathit{\psi })$, we only need to show that {ℓ̇₂(t, , ψ) − ℓ̇₂(t, , ψ₀): ||ψ|| < δ, t ∈ [0, τ]} is P₀-Donsker, where

$$\begin{array}{l}{\stackrel{.}{\ell }}_2(t,\mathcal{O},\mathit{\psi })-{\stackrel{.}{\ell }}_2(t,\mathcal{O},{\mathit{\psi }}_0)={\int }_0^{t}Y(u)e^{{\mathit{\beta }}_0^{\prime }\mathit{Z}}d{\mathrm{\Lambda }}_0(u)-{\int }_0^{t}Y(u)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)\\ +{\int }_0^t\left\{{\int }_u^{\tau }S(v\mid \mathit{Z})dv\right\}{e}^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)/{\mu }_{\mathit{Z}}(\mathit{\psi })-{\int }_0^t\left({\int }_u^{\tau }{S}_0(v\mid \mathit{Z})dv\right)e^{{\mathit{\beta }}_0^{\prime }\mathit{Z}}d{\mathrm{\Lambda }}_0(u)/{\mu }_{\mathit{Z}}({\mathit{\psi }}_0).\end{array}$$

First, the class of functions exp(β′ Z) for β in the compact set is P₀-Donsker as it is of finite dimension. The class of functions of bounded variation on [0, τ] is P₀-Donsker (van der Vaart, 1998, page 273). As the function Λ (t) is of bounded variation on [0, τ], the set of functions { ${\int }_0^{t}Y(u)e^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }$, t ∈ [0, τ], β, Λ} is P₀-Donsker. Similarly, we can show that the sets of functions { ${\int }_0^t[{\int }_u^{\tau }S(v\mid \mathit{Z})dv]e^{{\mathit{\beta }}^{\prime }\mathit{Z}}d\mathrm{\Lambda }(u)$, t ∈ [0, τ], β ∈ , Λ ∈ } and {μ_Z(β, Λ), β ∈ , Λ ∈ } are P₀-Donsker. Note that their envelope functions are (τ − u)e^β′ZΛ (τ) and τ < ∞, respectively. By Assumption 4, (τ − u)²Λ²(τ)E₀e^2β′Z < ∞ for all β and Λ ∈ . By the permanent property of Donsker classes, we proceed to confirm the Donsker property by using the fact that the sums, production and Lipschitz transformations of simple P₀-Donsker classes are still P₀-Donsker.

Furthermore, ||ψ − ψ₀|| → 0, ℓ̇₂(t, , ψ) converges to ℓ̇₂(t, , ψ₀) for each t. The convergence also holds in the square moment by the dominated convergence theorem. It follows that

$$\underset{t\in [0,\tau ]}{sup}{E}_0{\left|\right|\stackrel{.}{\mathit{\ell }}(t,\mathcal{O},\mathit{\psi })-\stackrel{.}{\mathit{\ell }}(t,\mathcal{O},{\mathit{\psi }}_0)\left|\right|}_{\mathcal{H}}^2\to 0.$$

The stochastic approximation of $\sqrt{n}({\mathit{U}}_n-{\mathit{U}}_0)({\widehat{\mathit{\beta }}}_n,{\widehat{\mathrm{\Lambda }}}_n)$ to $\sqrt{n}({\mathit{U}}_n-{\mathit{U}}_0)({\mathit{\beta }}_0,{\mathrm{\Lambda }}_0)$ now follows by a technical lemma of van der Vaart and Wellner (1996, Lemma 3.3.5, p. 311).