Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2008 Dec 11.
Published in final edited form as: Biometrika. 2008 Dec;95(4):947–960. doi: 10.1093/biomet/asn049

Semiparametric Maximum Likelihood Estimation in Normal Transformation Models for Bivariate Survival Data

Yi Li 1, Ross L Prentice 2, Xihong Lin 3
PMCID: PMC2600666  NIHMSID: NIHMS50270  PMID: 19079778

SUMMARY

We consider a class of semiparametric normal transformation models for right censored bivariate failure times. Nonparametric hazard rate models are transformed to a standard normal model and a joint normal distribution is assumed for the bivariate vector of transformed variates. A semiparametric maximum likelihood estimation procedure is developed for estimating the marginal survival distribution and the pairwise correlation parameters. This produces an efficient estimator of the correlation parameter of the semiparametric normal transformation model, which characterizes the bivariate dependence of bivariate survival outcomes. In addition, a simple positive-mass-redistribution algorithm can be used to implement the estimation procedures. Since the likelihood function involves infinite-dimensional parameters, the empirical process theory is utilized to study the asymptotic properties of the proposed estimators, which are shown to be consistent, asymptotically normal and semiparametric efficient. A simple estimator for the variance of the estimates is also derived. The finite sample performance is evaluated via extensive simulations.

Keywords: Asymptotic normality, Bivariate failure time, Consistency, Semiparametric efficiency, Semiparametric maximum likelihood estimate, Semiparametric normal transformation

1 Introduction

Examples of censored bivariate data include the Danish Twin Study (Wienke et al., 2002), the diabetic retinopathy study (Hougaard, 2000), the dual infection kidney dialysis study (Van Keilegom & Hettmansperger, 2002) and the reproductive health study of the association of age at a marker event and age at menopause (Nan et al., 2006). In all these studies, the assessment of marginal distribution as well as dependence among dependent individuals, such as twins, is of major interest, the latter because it renders genetic information.

Few existing bivariate distributions for nonnegative random variables accommodate semiparametric specifications of marginal distribution and unrestricted pairwise dependence. Consider Clayton's (1978) model for a pair of survival times (T˜1,T˜2), defined by

S(t˜1,t˜2)=[max{S1(t˜1)θ+S2(t˜2)θ1,0}]θ1, (1)

where S(t˜1,t˜2)=pr(T˜1>t˜1,T˜2>t˜2),S1(t˜1)=S(t˜1,0)andS2(t˜2)=S(0,t˜2) are bivariate survival and marginal survival functions respectively, and θ has an interpretation as cross-ratio (Oakes, 1989) as well as corresponding to other dependence measures such as Kendall's τ. This model allows for negative dependence when −1 < θ < 0. However, for random variables T˜1 and T˜2 which are marginally absolutely continuous with respect to the Lebesgue measure µ, the joint distribution of (T˜1,T˜2) is absolutely continuous with respect to the product Lebesgue measure µ × µ only when θ > −0.5. When θ ≤ −0.5, Oakes (1989) noted that the distribution is no longer absolutely continuous, but has a mass along the curve given by {(t˜1,t˜2):S1(t˜1)θ+S2(t˜2)θ1=0}. Hougaard (2000) further noted that frailty models cannot yield unrestricted marginal distributions with unrestricted pairwise parameters.

Hence it is of substantial interest to specify a semiparametric likelihood model that allows for arbitrary modelling of the marginal survival functions, that allows for a flexible and interpretable correlation structure, and that retains a likelihood so that an efficient and simple estimating procedure is possible. For this purpose, we study a class of semiparametric normal transformation models for right censored bivariate failure times. Specifically, nonparametric marginal hazard rate models are transformed to a standard normal model and a joint normal distribution is imposed on the bivariate vector of transformed variates. The induced joint distribution is closely related to the normal copula model developed by, for example, Klaassen & Wellner (1997) and Pitt et al. (2006). However, all the previous efforts in normal copula focused only on non-censoring situations, and it is unclear whether these existing results can be generalized to censoring situations.

This paper is motivated by a recent work of Li & Lin (2006) on spatial survival data. Li & Lin only considered estimating equation approaches in spatial settings and their estimators are not efficient under the bivariate normal transformation model. In contrast, we focus this paper on semiparametric likelihood based inference for bivariate survival data.

2 Semiparametric Normal Transformation Models

Consider a pair of survival times (T˜1,T˜2), where each T˜j marginally has a cumulative hazard Λj(t). Then Λj(T˜j) marginally follows a unit exponential distribution, and its probit transformation

Tj=Φ1{1eΛj(T˜j)} (2)

has a standard normal distribution, where Φ(·) is the cumulative distribution function for N(0, 1).

To specify the correlation structure within the survival time pair (T˜1,T˜2), we assume that the normally transformed survival time pair (T1; T2) is jointly normally distributed with correlation coefficient ρ and with a joint tail probability function

Ψ(z1,z2;ρ)=z1z2ϕ(x1,x2;ρ)dx1dx2 (3)

where ϕ(x1, x2; ρ) is the joint probability density function for a bivariate normal vector with mean (0, 0) and covariance matrix (1ρρ1). It follows that the bivariate survival function for the original survival time pair (T˜1,T˜2) is

S(t˜1,t˜2;ρ)=pr(T˜1>t˜1,T˜2>t˜2;ρ)=Ψ[Φ1{F1(t˜1)},Φ1{F2(t˜2)};ρ], (4)

where Fj(·) is the marginal cumulative distribution function of T˜j for j = 1, 2. In addition, the density for the original survival time pair (T˜1,T˜2) is

f(t˜1,t˜2;ρ)=f1(t˜1)f2(t˜2)eg(t1,t2;ρ), (5)

where ti=Φ1{1eΛi(t˜i)},fi(t˜)=λi(t˜)exp{Λi(t˜)} is the marginal density for T˜i,i=1,2 and

g(t1,t2;ρ)=0.5log(1ρ2)0.5(1ρ2)1(ρ2t12+ρ2t222ρt1t2). (6)

It is obvious that ρ = 0 results in f(t˜1,t˜2;ρ=0)=f1(t˜1)f2(t˜2), corresponding to the independent case. One can easily show that the bivariate survival function approaches the upper Fréchet bound min{S1(t˜1),S2(t˜2)}asρ1, and the lower Fréchet bound max{S1(t˜1)+S2(t˜2)1,0}asρ1+. Indeed, the correlation parameter ρ provides a summary measure for the pairwise dependence, whose connection with the other commonly used dependence measures, including Kendall's tau, Spearman’s rho and the cross ratio, can be found in Li & Lin (2006). Also of interest is to note that (5) can be rewritten as

f(t˜1|t˜2)f(t˜1|O)=f(t˜2|t˜1)f(t˜2|O)=eg(t1,t2;ρ),

where f(·|·) denotes a conditional density function and O is the empty set. Hence, function g or parameter ρ can also be interpreted as a Bayes factor for a dependence model against an independence model.

We are now in a position to consider estimation based on a censored sample of m pairs; that is, we estimate the marginal hazard rate and the correlation parameter on the basis of observed pairs (X˜i1,δi1,X˜i2,δi2), where X˜ij=T˜ijU˜ijmin(T˜ijandU˜ij),δij=I(T˜ijU˜ij),forj=1,2. For simplicity, we assume that censoring is random in that the censoring pair (U˜i1,U˜i2) is independent of the survival pair (T˜i1,T˜i2). The likelihood function can then be factorized into the product of contributions from the survival and censoring times, facilitating likelihood-based inference.

In some applications involving bivariate survival data, including studies of disease occurrence patterns of twins or siblings, it is natural to restrict the marginal cumulative hazard to be common for members of the same pair. Hence, we first consider drawing inference with the constraint of Λ1 ≡ Λ2 in §3, followed by the case of separate marginal cumulative hazards Λj, j = 1, 2, which do not have such a constraint, in §4.

3 Semiparametric Maximum Likelihood Estimation With A Common Marginal Cumulative Hazard

3.1 The Likelihood Function

This section proposes a semiparametric maximum likelihood estimation procedure for the semiparametric normal transformation model with a common marginal cumulative hazard, Λ, say. We define the normally transformed observed time Xij=Φ1{1exp(Λ(X˜ij)},forj=1,2. As this transformation is monotone, it can easily accommodate right censored data as the transformed outcome (Xij, δij) contains the same information as the original (X˜ij,δij), facilitating the derivation of a likelihood function that can be factorized into the product of contributions from the survival and censoring times. It follows that the likelihood function for the unknown parameters (Λ, ρ) can be written, up to a constant, as the product of factors

L˜i(ρ,Λ)={eg(Xi1,Xi2;ρ)Λ′(X˜i1)Λ′(X˜i2)eΛ(X˜i1)Λ(X˜i2)}δi1δi2{Ψ1(Xi1,Xi2;ρ)Λ′(X˜i1)eΛ(X˜i1)}δi1(1δi2)  ×{Ψ2(Xi1,Xi2;ρ)Λ′(X˜i2)eΛ(X˜i2)}(1δi1)δi2×{Ψ(Xi1,Xi2;ρ)}(1δi1)(1δi2), (7)

i = 1,…,m where Ψj(x1,x2;ρ)=(/xj)Ψ(x1,x2;ρ)/ϕ(xj),forj=1,2. Indeed, Ψj(x1,x2;ρ)=pr(T3jx3j|Tj=xj),forj=1,2.

Direct maximization of the above likelihood in a space containing continuous hazard Λ(·) is not feasible, as one can always let the likelihood go to ∞ by choosing some continuous function Λ(·) with fixed values at each X˜ij while letting Λ′(·) go to ∞ at an observed failure time, for example, at some X˜ij with δij = 1. Thus we need to assume that Λ is cadlag and piecewise constant, where by cadlag we mean right continuous with left hand limit. It follows that the maximum likelihood estimator of Λ(·) will be the one which jumps only at distinct observed failure times. We denote the jump size of Λ(·) at t by ΔΛ(t) = Λ(t) − Λ(t−). The semiparametric maximum likelihood estimator is the maximizer of the empirical likelihood function L(ρ, Λ), which is the product of terms (7) with Λ′(·) replaced by ΔΛ(·). We denote the log empirical likelihood function by ℓ(ρ, Λ) = log L(ρ, Λ).

3.2 Theoretical Properties of the Semiparametric Maximum Likelihood Estimator

The main results of the paper are proved under the following set of regularity conditions. Condition 1. (Boundedness.) The parameter ρ lies in a compact set within (−1, 1). Condition 2. (Finite Interval.) There exist a t0 > 0 and a constant c0 > 0 such that pr(U˜ijt0)=pr(U˜ij=t0)>c0. In practice, t0 is usually the duration of the study. Condition 3. (Differentiability.) The marginal cumulative hazard Λ (t) is differentiable and Λ′(t) > 0 over [0, t0]. Moreover, Λ(t0) < ∞.

Condition 1 ensures the existence and consistency of the estimators. A similar boundedness condition on the frailty parameter was assumed by Murphy (1994) in the context of frailty models for the same purpose. Condition 2 ensures that the failures for both pair members can be observed over a finite interval [0, t0], which makes it possible to estimate the hazard over [0, t0]. Condition 3 implies absolute continuity of the cumulative hazard, which is useful in the consistency proof, and that we can work with the supremum norm on the space of cumulative hazard functions. Also, Condition 3 guarantees the identifiability of the semiparametric normal transformation model specified in (2) and (3).

Under Conditions 1–3, we show in a technical report, obtainable at http://biowww.dfci.harvard.edu/~yili/bikaproof.pdf, that the semiparametric maximum likelihood estimates do exist and are finite. Furthermore, the next two Propositions indicate that the semiparametric maximum likelihood estimator of Λ remains bounded, and that the estimators of {ρ,Λ(·)} are consistent and asymptotically normal estimators of the true parameters. The proofs can be found in the aforementioned technical report.

Proposition 1

(Consistency) Denote the true parameters by0, Λ0). Then |ρ̂ − ρ0| → 0 and supt[0,t0]|Λ^(t)Λ0(t)|0 almost surely.

Proposition 2

(Asymptotic Normality) The scaled process m1/2(ρ̂−ρ0,Λ̂−Λ0) converges weakly to a zero-mean Gaussian process in the metric space R×l[0, t0], where l[0, t0] is the linear space containing all the bounded functions in [0, t0] equipped with the supremum norm. Furthermore, ρ̂ and 0t0η(s)dΛ^(s) are asymptotically efficient, where η(s) is any function of bounded variation over [0, t0].

Proposition 2 implies that both ρ̂ and Λ̂(t), and hence the estimator of the marginal survival are asymptotically efficient by taking η(s) = I(st) for any t ∈ [0,t0]. It further implies that the infinite-dimensional parameter, Λ(·), can be treated in the same fashion as the finite-dimensional correlation parameter ρ. Hence the asymptotic covariance matrix can be estimated by inverting the observed information matrix. To be specific, for any constant h1 and any function h2 of bounded variation, the asymptotic variance of

h1ρ^+0t0h2(s)dΛ^(s)=h1ρ^+{(i,j):δij=1}h2(X˜ij)ΔΛ^(X˜ij) (8)

can be estimated by h′^J^1h^, where ĥ is a column vector comprising h1 and h2(X˜ij) for which δij = 1, and Ĵ is the negative Hessian matrix of ℓ(ρ, Λ) with respect to ρ and the jump size of Λ at X˜ij when δij = 1. More formally, it can be shown that mh′^J^1h^V(h1,h2) in probability as m → ∞, where V (h1; h2) is the asymptotic variance of m1/2{h1ρ^+0t0h2(s)dΛ^(s)}. The justification follows the proof of Theorem 3 in Parner (1998), who argued that the empirical information operator based on Ĵ approximates the true invertible information operator. We will evaluate the finite-sample performance of this variance estimator in §5.

3.3 A Positive-Mass-Redistribution Algorithm

Consider the following computationally efficient procedure for obtaining the semiparametric maximum likelihood estimates and their variance-covariance matrix. Since T˜1 and T˜2 have the same distribution function, whose estimator has masses at the distinct failure times of T˜1 and T˜2, we denote by t1 < … < tK the K distinct, ordered and pooled T˜1 and T˜2 failure times. Define r(t1,t2)=#{l|X˜1lt1,X˜2lt2} to be the size of the risk set at (t1, t2) and let R˜={(t1,t2)|r(t1,t2)>0} denote the risk region. We focus on the square grids {(t1,t2)|t1=ti,t2=tj,1iK,1jK} formed by the observed T˜1 and T˜2 pooled failure times. This is because the censored values in T˜1, or T˜2, in the sample can be replaced by censored values at the immediately smaller T˜1 and T˜2 pooled uncensored failure time, or by zero if there is no corresponding smaller uncensored time, without affecting the log empirical likelihood ℓ(ρ, Λ). We refer to such replacement as positive mass redistribution. Then let nijδ1δ2=#{l|X˜1l=ti,X˜2l=tj,δ1l=δl,δ2l=δ2}forδ1,δ2{0,1}, 0 ≤ i K and 0 ≤ jK, with t0 = 0. Also let fij = f(ti,tj), Fi=l=1i(1λl) and Fi=Fi1, for i = 1,…,K, where λl = ΔΛ(tl). The log empirical likelihood ℓ(ρ, Λ) defined in §3.1 can now be written as

=i=1Kj=1Knij11logflm+nij10log(λiFiυ=1jfiυ)+nij01log(λjFju=1ifuj) +nij00log(Fi+Fj+u=1iυ=1jfuυ1), (9)

which involves only the marginal hazard rates at uncensored T˜1 and T˜2 times, and the joint density at grid points in the risk region. The latter can be rewritten as

fij=λiFiλjFjeg(si,sj;ρ),

with g(·,·) defined in (6) and si = Φ−1(1 − Fi). To ensure numerical stability and avoid arguments of 0 for Φ−1 in computation in finite samples, we use an asymptotically equivalent transformation si = Φ−1{1 − (1 − m−1)Fi}. A simple Newton-Raphson procedure, starting with ρ = 0 and the Kaplan-Meier marginal hazard rates derived by treating (X˜ij,δij), j = 1, 2, i = 1,…, m, as 2m independent observations, can be used to compute the semiparametric maximum likelihood estimates λ^i, ρ̂. These calculations are less computationally demanding as they do not require the evaluation of bivariate incomplete normal integrals, only the evaluation of the univariate Φ−1.

If we follow the arguments in §3.2, the variability of λ^i, ρ̂ can be assessed by inverting the negative Hessian matrix of (9), denoted by Ĵ, a (K + 1) × (K + 1) matrix. Furthermore, the functional (8) can be rewritten as a linear combination of ρ̂ and the λ^i, namely,

h1ρ^+i=1Kh2(ti)λ^i,

whose variance function can be easily computed as h′^J^1h^, where ĥ = {h1, h2(t1),…,h2(tK)}′. We can easily apply this result to estimate the variance of the estimator of a survival probability. For example, the common marginal survival S(u0)=pr(T˜1>u0) at any given time u0 ∈ [0,t0] can be estimated by S^(u0)=eΛ^(u0). With a first-order Taylor expansion,

S^(u0)S(u0)S(u0){Λ^(u0)Λ(u0)}=0t0S(u0)I(su0)dΛ^(s)+const

Hence, Ŝ(u0) can be approximated by the functional form (8) with h1 = 0 and h2(s) = −S(u0)I(su0) and application of the above results will render S^2(u0)e′^J^1e^ as a consistent estimator of the variance of Ŝ(u0), where ê = {0; I(t1u0),…,I(tKu0)}′.

4 Semiparametric Maximum Likelihood Estimation for the Stratified-Hazard Model

4.1 The Estimator and Its Theoretical Properties

In this section, we relax the condition of a common marginal hazard and allow each T˜ij to have a separate cumulative hazard function Λj(·), j = 1, 2. This is often appropriate in practice.

We consider joint maximum likelihood estimation, following a development parallel to that for the common-hazard model. Our inference stems from the loglikelihood function of unknown parameters (Λ1, Λ2, ρ) based on the observed data (X˜ij,δij), j = 1, 2, i = 1,…, m, which can be written, up to a constant, as the product over i = 1,…,m of terms

 L˜i(ρ,Λ1,Λ2)={eg(Xi1,Xi2;ρ)Λ1(X˜i1)Λ2(X˜i2)eΛ1(X˜i1)Λ2(X˜i2)}δi1δi2{Ψ1(Xi1,Xi2;ρ)Λ1(X˜i1)eΛ1(X˜i1)}δi1(1δi2) ×{Ψ2(Xi1,Xi2;ρ)Λ2(X˜i2)eΛ2(X˜i2)}(1δi1)δi2×{Ψ(Xi1,Xi2;ρ)}(1δi1)(1δi2). (10)

Here Xij=Φ1{1exp(Λj(X˜ij)} for j = 1, 2. Again, direct maximization of the likelihood function in a space containing continuous hazards Λ1(·) or Λ2(·) is infeasible, as one can always make the likelihood arbitrarily large by constructing some continuous functions Λ1(·) and Λ2(·) with fixed values at each X˜ij while letting Λ1(·) or Λ2(·) go to ∞ at an observed failure time. Hence, when performing the maximum likelihood estimation, we assume that (Λ1, Λ2) are cadlag and piecewise constant. It follows that the semiparametric maximum likelihood estimator, (ρ^,Λ^1,Λ^2), is the maximizer of the empirical likelihood function ℓ(ρ, Λ1, Λ2), which is obtained from (10) with the derivatives Λ1(·) and Λ2(·) at the observed failure times respectively replaced by their jumps ΔΛ1(·) and ΔΛ2(·) at the corresponding time-points. We can show that (ρ^,Λ^1,Λ^2) exist and are finite. Furthermore, under Conditions 1–3, with both Λ1 and Λ2 satisfying Condition 3, the asymptotic properties of the estimators are summarized in the following two theorems, the proofs of which can be found in our technical report.

Proposition 3

(Consistency) Denote the true parameters by0, Λ01, Λ02). Then |ρ̂ − ρ0| → 0, supt[0,t0]|Λ^1(t)Λ01(t)|0 and supt[0,t0]|Λ^2(t)Λ02(t)|0 almost surely.

Proposition 4

(Asymptotic Normality) The empirical process m1/2(ρ^ρ0,Λ^1Λ01,Λ^2Λ02) converges weakly to a zero-mean Gaussian process in the metric space R × ℓ[0, t0] × l [0; t0], where [0, t0] is the linear space containing all the bounded functions in [0, t0] equipped with the supremum norm. Furthermore, ρ̂, 0t0η1(s)dΛ^1(s) and 0t0η2(s)dΛ^2(s) are asymptotically efficient, where η1(s) and η2(s) are any functions of bounded variation over [0, t0].

As in the case of a common-hazard model, the asymptotic covariance matrix of the estimators of the unknown, finite-dimensional and infinite dimensional, parameters can be estimated by inverting the observed information matrix. For any constant h1 and any function h2 and h3 of bounded variation, the asymptotic variance of

h1ρ^+0t0h2(s)dΛ^1(s)+0t0h3(s)dΛ^2(s)=h1ρ^+{i:δi1=1}h2(X˜i1)ΔΛ^(X˜i1)+{i:δi2=1}h2(X˜i2)ΔΛ^2(X˜i2) (11)

can be estimated by h′^J^1h^, where ĥ is a column vector comprising h1, the h2(X˜i1) for which δi1 = 1 and the h2(X˜i2) for which δi2 = 1, and Ĵ is the negative Hessian matrix of ℓ(ρ,Λ1, Λ2) with respect to ρ and the jump sizes of Λj at X˜ij when δij = 1. Indeed, following the proof of Theorem 3 in Parner (1998), one can show mh′^J^1h^V(h1,h2,h3) in probability as m → ∞, where V (h1, h2, h3) is the asymptotic variance of m1/2{h1ρ^+0t0h2(s)dΛ^1(s)+0t0h3(s)dΛ^2(s)}.

4.2 Practical Implementation for the Stratified-Hazard Model

Denote by t11 < … < t1I the I distinct ordered T˜1-failure times and by t21 < … < t2J the J distinct T˜2-failure times in the observed sample. As defined in §3.2, let r(t1, t2) be the size of the risk set at (t1, t2) and let R˜ be the risk region. We only consider the rectangular grids {(t1, t2)|t1 = t1i, t2 = t2j 1 ≤ iI, 1 ≤ jJ} formed by the observed T˜1 and T˜2 failure times. This is because the censored values in T˜1, or T˜2, in the sample can be replaced by censored values at the immediately smaller T˜1, or T˜2, uncensored failure times, or by zero if there is no corresponding smaller uncensored time, without affecting the empirical likelihood ℓ(ρ,Λ1, Λ2). With these replacements, or the so-called positive-mass-redistributions, let nijδ1δ2=#{l|X˜1l=t1i,X˜2l=t2j,δ1l=δ1,δ2l=δ2} for δ12 ∈ {0, 1} and for 0 ≤ iI and 0 ≤ jJ, with t10 = t20 = 0. Also let fij=f(t1i,t2j),F1i=l=1i(1λ1l),F1i=F1,i1,F2j=k=1j(1λ2k), and F2j=F2,j1, where λ1l = ΔΛ1(t1l), λ2k = ΔΛ2(t2k). The log empirical likelihood function can now be written as

=i=1Ij=1Jnij11logflm+nij10log(λ1iF1iυ=1jfij)+nij01log(λ2jF2ju=1ifuj) +nij00log(F1i+F2j+u=1iυ=1jfuυ1), (12)

which involves only the marginal hazard rates at uncensored T1 and T2 times, and the joint density at grid-points in the risk region, namely,

fij=λ1iF1iλ2jF2jeg(s1i,s2j;ρ),

with s1i = Φ−1(1 − F1i) and s2j = Φ−1(1 − F2j). In practice, we use an asymptotically equivalent transformation s1i = Φ−1{1 − (1 − m−1)F1i} and s2j = Φ−1{1−(1−m−1)F2j}. A simple Newton-Raphson procedure, starting with ρ = 0, and Kaplan-Meier marginal hazard rates λ1i=j=1J(nij11+nij10)/r(t1i,0),λ2j=i=1I(nij11+nij01)/r(0,t2j), can be used to compute the SPMLEs λ^1i, λ^2j and ρ̂. We again note that the likelihood evaluations are less computationally demanding, requiring only the computation of the univariate Φ−1.

Similarly, the variability of λ^1i, λ^2j and ρ̂ can be assessed by inverting the negative Hessian matrix of (12), denoted by the (I + J + 1) × (I + J + 1) matrix Ĵ. Moreover, the functional (11) can be rewritten as a linear combination of ρ̂ and the λ^1i, λ^2j, namely,

h1ρ^+i=1Ih2(t1i)λ^1i+j=1Jh3(t2j)λ^2j,

whose variance can be easily computed by h′^J^1h^, where ĥ = {h1, h2(t11),…, h2(t1I), h3(t21),…, h3(t2J}′.

As an illustration of this variance formula. Consider the bivariate survival estimator Ŝ(u0, υ0) S(u0, υ0) at any given time pair (u0, υ0) ∈ [0,t0]2. Based on the semiparametric normal transformation model, this is

S^(u0,υ0)=Ψ[Φ1{1eΛ^1(u0)},Φ1{1eΛ^2(υ0)};ρ^].

To evaluate the variance of Ŝ(u0, υ0), we perform a first-order Taylor expansion, yielding

 S^(u0,υ0)S(u0,υ0)γ1(u0,υ0)(ρ^ρ0)+γ2(u0,υ0){Λ^1(u0)Λ1(u0)}+γ3(u0,υ0){Λ^2(υ0)Λ2(υ0)}=γ1(u0,υ0)ρ^+0t0γ2(u0,υ0)I(su0)dΛ^1(s)+0t0γ3(u0,υ0)I(sυ0)dΛ^2(s)+const

where γ1(t1,t2)=Ψ(x1,x2;ρ)/ρ, γ2(t1,t2)=Φ1(x1,x2;ρ0)exp{Λ1(t1)}, γ3(t1,t2)=Φ2(x1,x2;ρ0)exp{Λ2(t2)} and xj=Φ1{1eΛj(tj)}. Hence, Ŝ(u0, υ0) can be approximated by the functional form (11) with h1 = γ1(u0, υ0), h2(s) = γ2(u0, υ0)I(su0), h3(s) = γ3(u0, υ0)I(s ≤ υ0), and application of the above variance formula renders a consistent estimator of the variance for Ŝ(u0, υ0), namely, h′^J^1h^, where h^={γ^1(u0,υ0),γ^2(u0,υ0)I(t11u0),,γ^2(u0,υ0)I(t1Iu0),γ^3(u0,υ0)I(t21u0),,γ^3(u0,υ0)I(t2Ju0)}andr^j(·,·) is obtained from γj(·,·), for j = 1, 2, 3, with all the unknown parameters replaced by their estimators. We evaluate the finite-sample performance of this variance estimator in §5.

5 Numerical Studies

5.1 Preamble

A series of simulation studies were performed to examine the properties of the proposed estimator and to compare it with the existing bivariate survivor estimators, including the Prentice-Cai (Prentice & Cai, 1992), Dabrowska (1988) and repaired nonparametric maximum likelihood (van der Laan, 1996; Moodie et al, 2005) estimators. The simulation set-up mimics those in Prentice et al. (2004). The marginal distributions of T˜1 and T˜2 were specified as unit exponential, and the censoring time U˜1 was taken to be an exponential variate with mean 0.5. Three special cases for U˜2 were considered: U˜2=, corresponding to no T˜2 censoring; U˜2=U˜1, corresponding to univariate censoring; U˜2 is independent of U˜1 and is an exponential variate with mean 0.5. A sample size of 120 pairs was considered with 1000 repetitions at a given configuration.

5.2 Finite sample performance under the correct model

We began by evaluating the finite sample performance of the semiparametric maximum likelihood estimator when the true model follows the semiparametric normal transformation model (4) with ρ = 0.5. We considered both the common-hazard model and the stratified-hazard model, but, as both models yielded similar results, we only report in Table 1 the simulation results for the common-hazard model. As efficient estimation of the common hazard function or the common marginal distribution function is of major interest under the common-hazard model, we report only the estimates of the marginal survival function at various time-points in Table 1. The sample averages of the estimates are very close to the true values, and the model-based standard errors, which were computed by applying the results of §3.3, match very well with the empirical standard errors up to the 3rd decimal point.

Table 1.

Averages and model based and empirical SEs of the SPMLEs under the semiparametric normal transformation model (4) with a common hazard function. The true values are: ρ = 0.5; S(0.1625) = 0.85; S(0.3566) = 0.70; S(0.5978) = 0.55.

t = 0.1625 t = 0.3566 t = 0.5978
Censoring ρ SEe SEm Ŝ(t) SEe Ŝ(t) SEe Ŝ(t) SEe
Censoring on T1 only 0.502 0.109 0.104 0.842 0.023 0.692 0.035 0.546 0.044
Univariate censoring 0.503 0.122 0.114 0.842 0.025 0.694 0.035 0.546 0.051
Bivariate censoring 0.493 0.141 0.138 0.846 0.027 0.697 0.042 0.555 0.055

SEe, empirical standard error; SEm, model-based standard error.

5.3 Finite sample performance under the misspecified model

We next considered the robustness of the SPMLE when the semiparametric normal transformation model was misspecified. The failure times were generated under the following bivariate Clayton model:

S(t˜1,t˜2)={S1(t˜1)θ+S2(t˜2)θ1}θ1, (13)

with θ = 4, implying a strong positive dependence between T˜1 and T˜2. We compared the performance of the semiparametric maximum likelihood estimator based on the semiparametric normal transformation model, S^NT with the other existing nonparametric estimators, including the Prentice-Cai estimator, S^PC, the empirical hazard rate estimator, S^E, and the redistributed empirical estimator, S^RE, which were taken from Table 1 and Table 2 of Prentice et al. (2004). As Prentice et al. (2004) only considered stratified-hazard models, we focused on the stratified-hazard model to make the resulting estimates comparable.

Table 2.

Averages, SEs and mean squared errors (MSE) for various bivariate survival function estimators at various time pairs (t1, t2) when the correct model (13) with θ = 4 is misspecified to the semiparametric normal model (4). The true bivariate survival probabilities at these pairs are 0.771, 0.666 and 0.608, respectively.

(t1, t2) = (0.1625, 0.1625) (t1, t2) = (0.1625, 0.3566) (t1, t2) = (0.3566, 0.3566)

Censoring bias SE MSE bias SE MSE Bias SE MSE
(×10−3) (×10−3) (×10−3)
Censoring on T1 only S^E 0.0% 0.046 2.1 0.3% 0.055 3.0 0.0% 0.057 3.2
S^PC 0.1% 0.040 1.6 0.1% 0.043 1.9 −0.1% 0.047 2.2
S^RE 0.1% 0.043 1.8 0.3% 0.046 2.1 0.0% 0.049 2.4
S^NT 3.2% 0.035 1.7 4.4% 0.035 2.0 3.3% 0.043 2.2
(0.033) (0.030) (0.038)
Univariate Censoring S^E 0.0% 0.051 2.6 0.3% 0.059 3.5 0.1% 0.063 4.0
S^PC 0.1% 0.041 1.7 0.3% 0.049 2.4 0.0% 0.051 2.6
S^RE 0.1% 0.048 2.3 0.3% 0.057 3.2 0.0% 0.058 3.4
S^NT 2.6% 0.032 1.4 3.3% 0.039 2.0 2.1% 0.042 1.9
(0.028) (0.037) (0.042)
Bivariate Censoring S^E 0.0% 0.058 3.4 0.3% 0.073 5.3 −0.1% 0.077 5.9
S^PC −0.1% 0.041 1.7 −0.1% 0.049 2.4 −0.3% 0.053 2.8
S^RE −0.2% 0.056 3.1 0.3% 0.066 4.4 −0.3% 0.068 4.6
S^NT 2.2% 0.035 1.5 1.9% 0.042 1.9 0.1% 0.047 2.2
(0.033) (0.046) (0.048)

S^E, the empirical hazard rate estimator; S^PC, the Prentice-Cai estimator; S^RE, the redistributed empirical estimator; S^NT, the semiparametric maximum likelihood estimator; SEs, empirical standard errors; for S^NT the model based standard errors are displayed inside the brackets.

Table 2 and Table 3 contain the sample averages of the relative biases of the bivariate survival estimates and marginal survival estimates at selected time-points and the average model-based standard errors, calculated by applying the results of §4.2, for the point estimates, along with the empirical standard errors. We also list the summary statistics for the empirical hazard rate, S^E, Prentice-Cai, S^PC, and redistributed empirical, S^RE estimators (Prentice et al., 2004). Finally, we computed the mean squared errors for all the estimators.

Table 3.

Averages and SEs for various estimators of the marginal survival functions S1 and S2 when the correct model (13) with θ = 4 is misspecified to the semiparametric normal model (4). The true marginal survival probabilities for both T1 and T2 are 0.850, 0.700 and 0.55, respectively.

t1 = 0.1625 t1 = 0.3566 t1 = 0.5978

Censoring bias SE MSE bias SE MSE Bias SE MSE
(×10−3) (×10−3) (×10−3)
Censoring on T1 only S^E 0.0% 0.036 1.3 0.0% 0.049 2.4 0.2% 0.063 3.9
S^KM 0.0% 0.036 1.3 0.1% 0.049 2.4 0.4% 0.064 4.1
S^RE 0.1% 0.036 1.3 0.2% 0.048 2.3 1.6% 0.060 3.7
S^NT 0.8% 0.031 1.0 2.7% 0.047 2.5 3.4% 0.065 4.6
(0.036) (0.050) (0.062)
Univariate censoring S^E 0.1% 0.043 1.8 0.1% 0.057 3.2 0.4% 0.071 5.0
censoring S^KM 0.0% 0.036 1.3 0.1% 0.049 2.4 0.4% 0.064 4.1
S^RE 0.1% 0.041 1.7 0.3% 0.054 2.9 1.1% 0.069 4.8
S^NT 0.6% 0.030 0.9 2.4% 0.046 2.4 3.4% 0.061 4.1
(0.035) (0.050) (0.067)
Bivariate S^E −0.1% 0.048 2.3 0.1% 0.072 5.1 −0.4% 0.100 10.0
censoring S^KM 0.0% 0.035 1.2 0.3% 0.051 2.6 0.4% 0.064 4.1
S^RE 0.0% 0.047 2.2 0.6% 0.065 4.2 2.3% 0.069 4.9
S^NT 0.9% 0.031 1.0 1.9% 0.047 2.4 1.0% 0.063 4.0
(0.035) (0.051) (0.065)

S^E, the empirical hazard rate estimator; S^KM, the Kaplan-Meier estimator; S^RE, the redistributed empirical estimator; S^NT, the semiparametric maximum likelihood estimator. SEs; empirical standard errors; for S^NT the model based standard errors are displayed inside the brackets; only the estimates of the T1-survival are reported, as similar patterns were observed for the estimates of the T2-survival.

Our results show that, even when the underlying model is misspecified, the semiparametric maximum likelihood estimation based on the semiparametric normal transformation model incurred only small biases. Among all the scenarios examined, the relative biases, i.e. (point estimate - true value)/true value, of the semiparametric maximum likelihood estimates ranged from −5.7% to 4%. Compared to the competing nonparametric estimators, the new estimator also achieved high efficiency and had the smallest standard errors in all most all the scenarios examined. It had a smaller MSE than the other estimators in most cases considered. In addition, the model-based standard errors were in a good agreement with their empirical counterparts.

5.4 Further comparison with inverse-probability-of-censoring-weighted estimator in efficiency and robustness

We also compared the new estimator with a doubly-robust inverse-probability-of-censoring-weighted estimator, derived under univariate censoring, which stipulates that the censoring time is common for both pair members (Lin & Ying, 1993; Tsai & Crowley, 1998; Wang & Wells, 1998; Nan et al., 2006). The detailed derivation can be found in our technical report. We first compared the efficiency of the inverse-probability-of-censoring-weighted estimator with the semiparametric maximum likelihood estimator when the true underlying model indeed followed the semiparametric normal transformation model (4) with ρ = 0.5. The results are documented in Table 4, which demonstrate that the new estimator has noticeably smaller variances than its competitor. We next considered the robustness of the inverse-probability-of-censoring-weighted estimator when the underlying model model was misspecified as the semiparametric normal transformation model, while the true model followed the bivariate Clayton model (13) with θ = 4. The results in Table 4 indicate that the inverse-probability-of-censoring-weighted estimator has eliminated the bias caused by the misspecification of the semiparametric normal transformation model, while the new estimator incurs negligible biases and retains smaller variances. In terms of mean squared error (MSE), the semiparametric maximum likelihood estimator was superior in the scenarios considered.

Table 4.

Comparison of the semiparametric normal transformation model (4) based SPMLE estimator and the IPCW estimator at various time pairs (t1, t2) under univariate censoring. The true underlying models are the semiparametric normal transformation model (4) with ρ = 0.5, i.e. the working model is correctly specified, and Clayton’;s model (13) with θ = 4, i.e. the working model is misspecified. The true bivariate survival probabilities at the specified points are 0.7577, 0.6415 and 0.5568, respectively, and the averages of the relative biases are listed in the table.

True Underlying Model (t1; t2) = (0.1625, 0.1625) (t1; t2) = (0.1625, 0.3566) (t1; t2) = (0.3566, 0.3566)

bias SEe MSE bias SEe MSE bias SEe MSE
(×10−3) (×10−3) (×10−3)
Semiparametric Normal S^NT 0.1% 0.040 1.60 0.0% 0.049 2.40 0.1% 0.050 2.50
Transformation Model S^IP −0.1% 0.043 1.85 −0.1% 0.052 2.70 0.0% 0.055 3.03

Clayton Model S^NT 2.6% 0.031 1.41 3.3% 0.039 1.98 2.1% 0.042 1.90
S^IP 0% 0.041 1.68 0.3% 0.051 2.61 0% 0.053 2.81

S^NT, the semiparametric maximum likelihood estimator based on the semiparametric normal transformation model; S^IP, the inverse-probability-of-censoring-weighted estimator; SEe, the empirical standard error; MSE, the mean squared error.

5.5 Estimation of ρ

Finally, we discuss the interpretation and estimation of ρ, which has a one-one correspondence between the common dependence measure for bivariate survival, such as Kendall’s τ. As indicated in Li & Lin (2006),

τ=400f(t˜1,t˜2;ρ)S(t˜1,t˜2;ρ)dt˜1dt˜21,

where S(t˜1,t˜2;ρ) and f(t˜1,t˜2;ρ) are respectively the joint bivariate survival and density functions defined in (4) and (5). After a change of variables, we have that

ττ(ρ)=400Ψ(t1,t2;ρ)eg(t1,t2;ρ)ϕ(t1)ϕ(t2)dt1dt21,

where Ψ(·) is the joint tail function for the bivariate normal distribution defined in (3), g(t1, t2, ρ) is the ‘cross’ term defined in (6), and ϕ is the standard normal density function, none of which depends on any specific forms of hazard functions. As shown in Li & Lin (2006), ρ uniquely determines τ, and thus provides a standardized dependence measure for bivariate survival. Indeed, τ(ρ̂) yields the model-based estimator of Kendall's τ , with a model-based standard error that can be conveniently obtained using the delta method.

We considered the estimation and the interpretation of the estimate of ρ when the semiparametric normal transformation model was misspecified, and the failure times were generated under the bivariate Clayton model in (13). We chose θ to be 0.5, 1; 2 and 4, which correspond to Kendall’s τ of 0.199, 0.333, 0.5 and 0.613, respectively, using formula (4.4) of Hougaard (2000).

The sample averages of the estimates of ρ and the model-based Kendall’s τ are displayed in Table 5, along with the empirical and model-based standard errors. It appears that, when the underlying model is misspecified, the estimate of ρ itself might not be of interest, as it does not recover the specific dependence structure of the true model. However, the model-based estimator of Kendall’s τ based on the estimates of ρ are very comparable to the true values. We envisage that, at least for the scenarios we considered, the estimate of ρ would lead to a reasonable approximation for Kendall's τ even if the model were misspecified.

Table 5.

Averages and SEs of ρ and the model-based Kendall’ τ when the correct model (13) with various true θ is misspecified to the semiparametric normal model (4) with parameter ρ to estimate.

θ True Kendall’s Censoring ρ Model-based τ

τ ρ̂ SEe SEm tau SEe SEm
4 0.614 Censoring on T1 only 0.840 0.020 0.014 0.620 0.024 0.016
Univariate censoring 0.839 0.026 0.021 0.619 0.028 0.022
Bivariate censoring 0.820 0.033 0.030 0.608 0.033 0.026

2 0.500 Censoring on T1 only 0.696 0.042 0.038 0.490 0.035 0.030
Univariate censoring 0.685 0.063 0.057 0.487 0.050 0.041
Bivariate censoring 0.693 0.064 0.057 0.492 0.052 0.043

1 0.333 Censoring on T1 only 0.500 0.060 0.059 0.339 0.044 0.041
Univariate censoring 0.483 0.076 0.077 0.320 0.056 0.053
Bivariate censoring 0.493 0.077 0.072 0.327 0.058 0.051

0.5 0.199 Censoring on T1 only 0.313 0.067 0.072 0.209 0.046 0.048
Univariate censoring 0.291 0.084 0.078 0.194 0.054 0.062
Bivariate censoring 0.297 0.087 0.079 0.196 0.057 0.064

SEe, empirical standard errors; SEm, model-based standard errors,

6 Discussion

With the analytical framework established in this article, we are to extend the results to multivariate data, where clusters are allowed to have varying cluster sizes and where each pair of failure times may have a distinct correlation parameter. A key feature of this transformation model is that it can easily accommodate covariates in such a way that survival outcomes marginally follow a common Cox proportional hazards model, and their joint distribution is specified by a joint normal distribution. Hence, the regression coefficients have population-level interpretations, a feature not shared by conditional frailty models.

Acknowledgement

This work was supported in part by grants to each author from the U.S. National Institute of Health. The authors thank Professor D. M. Titterington and the associate editor for their insightful suggestions.

Contributor Information

Yi Li, Department of Biostatistics, Harvard School of Public Health and Dana-Farber Cancer Institute, 44 Binney Street, Boston, MA 02115, USA, yili@jimmy.harvard.edu.

Ross L. Prentice, Fred Hutchinson Cancer Research Center and University of Washington, 1959 NE Pacific Street, Seattle, WA 98195, rprentic@WHI.org

Xihong Lin, Department of Biostatistics, Harvard School of Public Health, 655 Huntington Avenue, Boston, MA 02115, xlin@hsph.harvard.edu.

References

  1. Hougaard P. Analysis of Multivariate Survival Data. New York: Springer-Verlag; 2000. [Google Scholar]
  2. van Keilegom I, Hettmansperger TP. Inference on multivariate M-estimators based on bivariate censored data. J. Am. Statist. Assoc. 2002;97:328–336. [Google Scholar]
  3. Klaassen CAJ, Wellner JA. Efficient estimation in the bivariate normal copula model: Normal margins are least favourable. Bernoulli. 1997;3:55–77. [Google Scholar]
  4. Li Y, Lin X. Semiparametric Normal Transformation Models for Spatially Correlated Survival Data. J. Am. Statist. Assoc. 2006;101:591–603. [Google Scholar]
  5. Lin D, Ying Z. A simple nonparametric estimator of the bivariate survival function under univariate censoring. Biometrika. 1993;80:573–581. [Google Scholar]
  6. Moodie FZ, Prentice RL. An adjustment to improve the bivariate survivor function repaired NPMLE. Lifetime Data Anal. 2005;11:291–307. doi: 10.1007/s10985-005-2964-9. [DOI] [PubMed] [Google Scholar]
  7. Murphy SA. Consistency in a proportional hazards model incorporating a random effect. Ann. Statist. 1994;22:712–731. [Google Scholar]
  8. Nan B, Lin X, Lisabeth LD, Harlow SD. Piecewise constant cross-ration estimation for association of age at a marker event and age at menopause. J. Am. Statist. Assoc. 2006;101:65–77. [Google Scholar]
  9. Oakes D. Bivariate survival models induced by frailties. J. Am. Statist. Assoc. 1989;84:487–493. [Google Scholar]
  10. Parner E. Asymptotic theory for the correlated gamma-frailty model. Ann. Statist. 1998;26:183–214. [Google Scholar]
  11. Pitt M, Chan D, Kohn R. Efficient Bayesian inference for Gaussian copula regression models. Biometrika. 2006;93:537–554. [Google Scholar]
  12. Prentice RL, Cai J. Covariance and survivor function estimation using censored multivariate failure time data. Biometrika. 1992;79:495–512. Correction (1993) 80, 711-2. [Google Scholar]
  13. Prentice RL, Hsu L. Regression on hazard ratios and cross ratios in multivariate failure time analysis. Biometrika. 1997;84:349–363. [Google Scholar]
  14. Prentice RL, Moodie ZF, Wu J. Hazard-based nonparametric survivor function estimation. J. R. Statist. Soc. B. 2004;66:305–319. [Google Scholar]
  15. Tsai W, Crowley J. A note on nonparametric estimators of the bivariate survival function under univariate censoring. Biometrika. 1998;85:573–580. [Google Scholar]
  16. van der Laan M. Efficient estimation in the bivariate censoring model and repairing NPMLE. Ann. Statist. 1996;24:596–627. [Google Scholar]
  17. Wang W, Wells M. Nonparametric estimators of the bivariate survival function under simplified censoring conditions. Biometrika. 1998;84:863–880. [Google Scholar]
  18. Wienke A, Lichtenstein P, Yashin A. A bivariate frailty model with a cure fraction for modeling familial correlations in diseases. Biometrics. 2003;59:1178–1183. doi: 10.1111/j.0006-341x.2003.00135.x. [DOI] [PubMed] [Google Scholar]

RESOURCES