Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Feb 8.
Published in final edited form as: Lifetime Data Anal. 2010 Apr;16(2):231–249. doi: 10.1007/s10985-009-9139-z

A copula model for bivariate hybrid censored survival data with application to the MACS study

Suhong Zhang 1, Ying Zhang 2,, Kathryn Chaloner 3, Jack T Stapleton 4
PMCID: PMC3567926  NIHMSID: NIHMS430987  PMID: 19921432

Abstract

A copula model for bivariate survival data with hybrid censoring is proposed to study the association between survival time of individuals infected with HIV and persistence time of infection with an additional virus. Survival with HIV is right censored and the persistence time of the additional virus is subject to interval censoring case 1. A pseudo-likelihood method is developed to study the association between the two event times under such hybrid censoring. Asymptotic consistency and normality of the pseudo-likelihood estimator are established based on empirical process theory. Simulation studies indicate good performance of the estimator with moderate sample size. The method is applied to a motivating HIV study which investigates the effect of GB virus type C (GBV-C) co-infection on survival time of HIV infected individuals.

Keywords: Association measure, Bivariate survival model, Copula, Current status data, Kendall's τ, Right censored data, Empirical process

1 Introduction and motivating example

This paper was motivated by the investigation of the association between survival time among HIV-infected subjects and co-infection with an additional apparently harmless virus named GB Virus Type C (or GBV-C). Several recent studies suggest that persistent co-infection of GBV-C is associated with prolonged HIV survival (for example, Xiang et al. 2001; Tillmann et al. 2001; Williams et al. 2004; Zhang et al. 2006), while this beneficial association was not significant in other studies (Toyoda et al. 1998; Birk et al. 2002).

Among all these studies, the Multicenter AIDS Cohort Study (MACS, Williams et al. 2004) is the most comprehensive study to date. It began to recruit subjects at risk for HIV infection from 1984, a time close to the beginning of the AIDS epidemic. For each subject, blood samples were taken and stored every 6 months. When diagnostic testing for HIV subsequently became available, seroconverters were identified through retrospective testing of the stored samples. Later, for a selected subset of seroconverters, two samples of stored blood were tested for GBV-C infection: one sample at 12–18 months after the subject's first positive HIV test (HIV onset), and the second was a sample at 4.5–6 years after seroconversion. The analysis conducted in Williams et al. (2004) treated all HIV survival times as right censored at January 1, 1996 to avoid confounding with the use of highly active HIV therapy that became available in 1996. They found that persistent GBV-C infection was significantly associated with prolonged survival among HIV-positive subjects at the late time (4.5–6 years after HIV onset), but not at the early time (12–18 months after HIV onset).

All previous studies compared the Kaplan-Meier survival curves between HIV-infected subjects with and without GBV-C infection at a specified time using the log-rank test. However, GBV-C viremia may clear over time and GBV-C persistence time varies among subjects. As a consequence, if GBV-C persistence time plays an essential role in its association with HIV survival then time to GBV-C clearance needs to be included in any comparison. This motivated the need to model GBV-C persistence time, rather than the status at a single time.

The use of Cox regression model with GBV-C status treated as a time-dependent covariate is not possible in this MACS data set. The Cox model requires that GBV-C status be known throughout the time during the study (Kalbfleisch and Prentice 2002, p. 200), but GBV-C status in the MACS study is only known at baseline and one another follow-up time.

In this paper we propose a bivariate survival model to adjust for the GBV-C persistence time since co-infection (time from HIV onset to the clearance of GBV-C). The GBV-C diagnostic test at the time close to HIV seroconversion is treated as the baseline GBV-C status, and the test at the second observation time provides current status data on GBV-C persistence time. Current status data, or interval censoring case 1 data, is a special case of interval censoring when it is only feasible to know whether or not an event (clearance) has occurred at a monitoring time (Groeneboom and Wellner 1992).

Bivariate and multivariate survival data have been studied extensively in the statistical literature. Liang et al. (1995) and Oakes (2000) reviewed some recent developments for analysis of multivariate failure time data. Copula based survival models are considered, for example, by Hougaard (1989), Oakes (1989), Shih and Louis (1995) and Wang and Ding (2000). Shih and Louis (1995) examined the association of the bivariate data that are both subject to right censoring, through a two-stage semiparametric estimation procedure. At the first stage in their procedure, the marginal survival functions are estimated consistently by nonparametric maximum likelihood estimators. At the second stage, a dependency structure is imposed by using a copula model, and the nonparametric maximum likelihood estimators of the two marginal survival functions are substituted into the likelihood function to form a pseudo-likelihood, then the association parameter is estimated through a pseudo-likelihood approach. Wang and Ding (2000) proposed a parallel two-stage semiparametric method for the bivariate current status data. In both papers, they showed that the proposed estimators of the association measure converge in distribution to normal random variables with the n1/2 rate without demonstrating the consistency first which is, however, required in the proof of asymptotic normality.

In this paper, we model the association of bivariate event times using a copula model and estimate the association parameter through the two-stage procedure as well. We focus specifically on the data structure where one of the paired event time data is right-censored and the other is observed as current status data, as observed in the MACS study. Our main goal in this paper is to develop an inference procedure to study the association of bivariate survival data with this type of censoring structure and to apply the proposed method to investigate the association between HIV survival and GBV-C persistence time.

The rest of this paper is organized as follows. Section 2 introduces a theoretical model and describes a two-stage semiparametric estimation procedure for the association parameter. Section 3 states asymptotic properties of the association parameter estimator. Section 4 presents simulation studies. Section 5 applies the proposed estimation method to the MACS GBV-C study. Finally, Section 6 summarizes the method with some remarks. Technical details are provided in the Appendix.

2 Likelihood and estimation method

In what follows the usual cumulative distribution function is defined as F(t) = P(Tt) and the corresponding survival function is defined as S(t) = P(T > t) = 1 – F(t).

Let T10 be the HIV survival time and T20 be the GBV-C persistence time. Assume the distributions of T10 and T20 are continuous. Let Sj and Fj, j = 1, 2 be the survival function and distribution function of Tj0, respectively. Denote F(t1, t2) and S(t1, t2) the joint distribution function and survival function of (T10,T20), respectively. We propose to model the joint survival function S(t1, t2) by the one-parameter Archimedean copula Cα:

Cα:[0,1]2[0,1]that satisfiesSα(t1,t2)=Cα(S1(t1),S2(t2)).

The joint distribution function (t1, t2) can therefore be expressed as Fα(t1, t2) = 1 – S1(t1) – S2(t2) + Sα(t1, t2).

Examples of various one-parameter Archimedean copula models are discussed in Nelsen (2006). As Kendall's τ is related to the Copula by τ = 4E {Cα(u, v)} – 1 (Nelsen 2006), the parameter α is naturally linked to the association between the two random variables with the marginal survival functions given by S1 and S2, respectively. Therefore, the inference for the association between the two event times can be made through the inference about α.

We consider bivariate survival data with hybrid censoring in which T10 is right censored by a random variable C1 and T20 is subject to interval censoring case 1 by a random monitoring time C2. Suppose we have collected a random sample of (T1i, T2i, Δ1i, Δ2i), i = 1, 2, . . . , n, from a distribution with density function f (t1, t2, δ1, δ2), where T1i=T1i0C1i and T2i=C2i;Δ1i=1[T1i0C1i] and Δ2i=1[T2i0C2i]. We consider the scenario of independent and non-informative censoring, i.e., (T10,T20) are jointly independent of (C1, C2), and the distribution of (C1, C2) is non-informative to any parameters in the joint distribution of (T10,T20). We also denote Gi(t) the marginal distribution function of Ci with density function gi(t), for i = 1, 2.

The density function f (t1, t2, δ1, δ2) can be explicitly written for four distinct cases with respect to Lebesgue measure. Combining the four cases and discarding the parts that are non-informative to the joint distribution of T10 and T20, we can derive the likelihood for n independently and identically distributed observations.

Let C1α(u,v)=uCα(u,v). Given the marginal survival functions S1 and S2, the likelihood for α, omitting parts that are irrelevant in estimating α, is

L(α,S1,S2;data)=i=1n[1C1α(S1(t1i),S2(t2i))]δ1iδ2i[C1α(S1(t1i),S2(t2i))]δ1i(1δ2i)×[S1(t1i)Cα(S1(t1i),S2(t2i))](1δ1i)δ2i[Cα(S1(t1i),S2(t2i))](1δ1i)(1δ2i). (1)

A two-stage maximum pseudo-likelihood estimation approach is developed to estimate α. The first stage involves the estimation of marginal survival functions for censored data. The marginal survival function S1 is estimated by the Kaplan-Meier estimator Ŝ1 and S2 is estimated by the nonparametric maximum likelihood estimator Ŝ2, using the Convex Minorant Algorithm described by Groeneboom and Wellner (1992).

At the second stage, Ŝ1(t) and Ŝ2(t) are substituted into the likelihood (1), the resulting pseudo-likelihood is then maximized with respect to α. The maximum pseudo-likelihood estimator α^n is the solution to the pseudo score equation:

Uα(α,S^1,S^2;data)=i=1nαl(α,S^1(t1i),S^2(t2i),δ1i,δ2i)=0, (2)

where

l(α,S^1(t1),S^2(t2),δ1,δ2)=δ1δ2log(1C1α(S^1(t1),S^2(t2)))+δ1(1δ2)logC1α(S^1(t1),S^2(t2))+(1δ1)δ2log(S^1(t1)Cα(S^1(t1),S^2(t2)))+(1δ1)(1δ2)logCα(S^1(t1),S^2(t2)). (3)

Note that the pseudo likelihood approach was previously adopted by Shih and Louis (1995) in an association study of bivariate right censored data and by Wang and Ding (2000) in a study of association between two event times with both subject to interval censoring case 1 (Groeneboom and Wellner 1992).

3 Asymptotic properties of the maximum pseudo-likelihood estimator α^n

Let T1 and T2 take values on [0, t01] × [0, t02], where t01 = sup {t : P(T1 > t, C1 > t) > 0} and t02 = sup {t : P(C2 > t) > 0}. Suppose α is in an open set A in the real line. Denote D a universal constant throughout the rest of technical development.

For the brevity of presentation, we define the following notations:

Vα(α,S1(t1),S2(t2),δ1,δ2)=αl(α,S1(t1),S2(t2),δ1,δ2)Vα2(α,S1(t1),S2(t2),δ1,δ2)=2α2l(α,S1(t1),S2(t2),δ1,δ2)Vα,1(α,S1(t1),S2(t2),δ1,δ2)=2αul(α,u,S2(t2),δ1,δ2)u=S1(t1)Vα,2(α,S1(t1),S2(t2),δ1,δ2)=2αvl(α,S1(t1),v,δ1,δ2)v=S2(t2)Vα2,1(α,S1(t1),S2(t2),δ1,δ2)=3α2ul(α,u,S2(t2),δ1,δ2)u=S1(t1)Vα2,2(α,S1(t1),S2(t2),δ1,δ2)=3α2vl(α,S1(t1),v,δ1,δ2)v=S2(t2)Vα,12(α,S1(t1),S2(t2),δ1,δ2)=3αu2l(α,u,S2(t2),δ1,δ2)u=S1(t1)Vα,1,2(α,S1(t1),S2(t2),δ1,δ2)=3αuvl(α,u,v,δ1,δ2)u=S1(t1),v=S2(t2)Vα,22(α,S1(t1),S2(t2),δ1,δ2)=3αv2l(α,S1(t1),v,δ1,δ2)v=S2(t2)

To study the asymptotic properties of α^n, we need the following regularity conditions. Some of the conditions are related to the smoothness of the copula models and the likelihood.

  • A1

    l(α, S1(t1), S2(t2), δ1, δ2) is three-time differentiable with respect to α on [0, t01] × [0, t02], for each αA, and all derivatives are continuous and uniformly bounded by some constant D.

  • A2

    Vα,1(α, S1(t1), S2(t2), δ1, δ2), Vα,2(α, S1(t1), S2(t2), δ1, δ2), Vα2,1(α, S1(t1), S2(t2), δ1, δ2), Vα2,2(α, S1(t1), S2(t2), δ1, δ2), Vα,12 (α, S1(t1), S2(t2), δ1, δ2), Vα,1,2(α, S1(t1), S2(t2), δ1, δ2), and Vα,22 (α, S1(t1), S2(t2), δ1, δ2) exist and are uniformly bounded by some constant D on [0, t01] × [0, t02], for all αA and survival functions S1 and S2.

  • A3

    For each αA, 0 < Eα[Vα (α, S1(T1), S2(T2), Δ1, Δ2)]2 < ∞.

  • A4

    F2 and G2 are absolutely continuous with respect to each other.

  • A5
    (ψ2g2)S21 is bounded and Lipschitz on [0, 1], where ψ2 is the derivative of the influence curve IC2(t2), defined by
    IC2(t2)=0t20t01Vα,2(α0,S1(τ1),S2(τ2),δ1,δ2)dP(τ1,τ2,δ1,δ2).
  • A6
    S2, g2 and ψ2 satisfy
    0t02S2(t2)(1S2(t2))g2(t2)ψ2(t2)dt2<.

Remarks

Conditions (A1) and (A2) require the log likelihood to be differentiable with respect to the unknown parameters. These conditions can be easily but tediously verified for Archimedean copulas. Condition (A3) indicates the log likelihood has finite nonzero information about α when the marginal survival functions are known which is usually required in parametric maximum likelihood theory. Conditions (A4)–(A6) are the regularity conditions given by Huang and Wellner (1995) in studying the asymptotic normality of linear functionals of the nonparametric maximum likelihood estimator of S2 with current status data. These regularity conditions are generally mild for applications.

The following two lemmas are important to study asymptotic properties of α^n.

Lemma 1 Let Fj={f:fisasurvivalfunctionon[0,t0j]}, j = 1, 2, and the class GF={Vα,1(α,f1(t1),f2(t2),δ1,δ2);fjFj,j=1,2}. Let P denote the probability measure of (T1, T2, Δ1, Δ2), then under condition (A1)–(A2), GF is a P-Glivenko-Cantelli class, for all α ∈ A.

Lemma 2 Let Fj={f:fisasurvivalfunctionon[0,t0j]}, j = 1, 2 and the class HF={Vα(α,f1(t1),f2(t2),δ1,δ2)Vα(α,S1(t1),S2(t2),δ1,δ2):fjFj,j=1,2}. Let P denote the probability measure of (T1, T2, Δ1, Δ2), then under condition (A1)–(A2), HF is a P-Donsker Class, for all α ∈ A.

Based on these two lemmas, the maximum pseudo-likelihood estimator α^n can be shown consistent and asymptotically normally distributed. The results are summarized in the following two theorems.

Theorem 1 Assume that the joint distribution of (T10,T20) follows an Archimedean copula model with the true association parameter α = α0. Under the regularity conditions (A1)–(A2), α^npα0 as n → ∞.

Theorem 2 Under the regularity conditions (A1)–(A6), n(α^nα0)dN(0,σ2), where

σ2=Var(Q(T1,T2,Δ1,Δ2;α0,S1,S2))W2(α0,S1,S2)

with

W(α0,S1,S2)=[Vα(α0,S1(t1),S2(t2),δ1,δ2)]2dP(t1,t2,δ1,δ2)Q(T1,T2,Δ1,Δ2;α0,S1,S2)=Vα(α0,S1(T1),S2(T2),Δ1,Δ2)+I1(T1,Δ1;α0)l~(T2,Δ2;S2,G2,ψ2),

in which

I1(T1,Δ1;α0)=0t010t02Mα,1(α0,S1(τ1),S2(τ2))f(τ1,τ2)I10(T1,Δ1)(τ1)dτ1dτ2andl~(T2,Δ2;S2,G2,ψ2)=[Δ2(1S2(T2))]ψ2(T2)g2(T2)I[g2(T2)>0],

where

Mα,1(α0,S1(t1),S2(t2))=E{Vα,1(α0,S1(T1),S2(T2),Δ1,Δ2)T1=t1,T2=t2}

and

I10(T1,Δ1)(t1)=S1(t1){0t11P(T1u)dN1(u)0t1I[T1u]P(T1u)dΛ1(u)}.

Here N1(u) is defined as I[T1u, Δ1 = 1] and Λ1 is the cumulative hazard function of T10.

The proofs of these lemmas and theorems are provided in the Appendix.

4 Simulation studies

Simulation studies are conducted to evaluate the finite sample performance of the proposed method. A Gumbel copula, a special case of Archimedean copulas, defined by

Cα(u,v)=exp{[(logu)α+(logv)α]1α},α1,0u,v1

is used to generate the bivariate event time data in which the two marginal distributions are both assumed to be exponential with unit rate 1. For the Gumbel copula, a larger α corresponds to a stronger positive association and α = 1 corresponds to the case that the two event times are independent.

A sample of bivariate copula random variables is generated based on their conditional distribution function. Suppose that the joint distribution of the bivariate data (T10,T20) is Cα(1 – exp(–t1), 1 – exp(–t2)). We generate (T10,T20) through the following steps:

  • – Generate two independent uniform (0, 1) random variables u, w.

  • – Set w = P(Vv|U = u) = ∂Cα(u, v)/∂u, solve for v.

  • – Set T10=log(1u), T20=log(1v).

Meanwhile, a sample of bivariate censoring times (C1 and C2) are each independently drawn from a uniform distribution on [0, 2.3]. In this setting, about 50% of T10 is right censored by C1, and about 50% of T20 is subject to interval censoring case 1 by C2 as well.

Kendall's τ is chosen as a global association measure. For the Gumbel copula, τ = 1 – 1/α. Three different values of α are set such that the corresponding Kendall's τ is 0.25, 0.5, and 0.75. For each value of α, we conduct Monte-Carlo simulations with 1,000 replications for sample size n = 50, 100, 200 and 400, respectively.

For each of the 1,000 simulations, Wald confidence interval is constructed based on the asymptotic normality, in which the standard error of α^n is computed using 200 bootstrap resamples. The empirical estimate of the coverage probability is obtained based on the Wald confidence interval over 1,000 replications.

Table 1 summarizes the simulation results for the two-stage pseudo-likelihood estimator. It provides results for estimation bias, Monte-Carlo standard deviation of 1,000 replicates as the empirical standard error (ese), mean of bootstrap standard error (bse), and 95% empirical coverage probability (ecp).

Table 1.

Simulation results of the two-stage maximum pseudo-likelihood estimator based on 1,000 Monte-Carlo samples with sample size ranged from 50 to 400 for α = 4/3, 2, 4

n = 50
n = 100
n = 200
n = 400
α^n
τ^n
α^n
τ^n
α^n
τ^n
α^n
τ^n
τ = 0.25 Bias 0.219 0.043 0.059 0.013 0.023 0.005 –0.005 –0.002
α = 1.333 ese 0.845 0.172 0.233 0.113 0.142 0.076 0.098 0.055
bse 8.799 0.161 0.334 0.109 0.154 0.077 0.099 0.054
95% ecp 0.968 0.966 0.963 0.954
τ = 0.50 Bias 1.120 0.051 0.194 0.021 0.102 0.014 0.032 0.003
α = 2.0 ese 8.090 0.158 0.563 0.098 0.320 0.070 0.208 0.050
bse 26.276 0.156 2.523 0.101 0.359 0.069 0.213 0.048
95% ecp 0.985 0.976 0.966 0.957
τ = 0.75 Bias 9.460 0.176 0.695 0.037 0.189 0.017 0.058 0.004
α = 4.0 ese 51.64 0.117 4.305 0.081 1.002 0.054 0.646 0.038
bse 60.48 0.124 15.726 0.079 1.597 0.054 0.696 0.038
95% ecp 0.991 0.980 0.974 0.959

As sample size increases, for a wide range of α, the biases of both α^n and τ^n decreases considerably, so do the Monte-Carlo standard deviation and bootstrap standard error. In addition, when sample size increases, the empirical coverage probability converges to the nominal level and the Monte-Carlo standard deviation and the mean of bootstrap standard error tend to get closer.

With same sample size, the stronger the dependency, the bigger the bias and the standard error for the estimator α^n, as greater variations are usually expected for larger values. Therefore, to preserve high efficiency, a larger sample size is desired to achieve reasonable performance of α^n when a strong association exists. Interestingly, we observe that the standard deviation of τ^n decreases as the association becomes stronger. This may be explained by the standard delta method which implies that στ^nσα^nα2 when sample size is large. The simulations demonstrate that this relationship approximately holds when n ≥ 200. We also note that the average of bootstrap standard error of estimated Kendall's τ is very close to the Monte-Carlo standard deviation when n ≥ 100, particularly if the association is not strong. This may imply the inference about the Kendall's τ will be reasonably good when n ≥ 100.

In addition to compute the proposed two-sage maximum pseudo-likelihood estimator α^n, we also compute α~n, the maximum likelihood estimator when the two marginal distributions are completely specified. The latter estimator serves as a benchmark to evaluate the performance of the maximum pseudo-likelihood estimator. Table 2 gives the results of α~n and τ~n, the maximum likelihood estimators of α and τ, respectively, when the two marginal survival functions are known. The maximum likelihood estimators perform better than the proposed maximum pseudo-likelihood estimators, as expected, but their differences are substantially reduced when sample size increases, say n ≥ 200. The small difference between the two estimators assures us the use of two-stage pseudo-likelihood estimation procedure, for which we gain the advantage of having flexibility by not modeling the marginal distributions but maintain high estimation efficiency with reasonable sample size.

Table 2.

Simulation results of maximum likelihood analysis (S1 and S2 are known) based on 1,000 Monte-Carlo samples with sample size ranged from 50 to 400 for α = 4/3, 2, 4

n = 50
n = 100
n = 200
n = 400
α~n
τ~n
α~n
τ~n
α~n
τ~n
α~n
τ~n
τ = 0.25 Bias 0.065 0.031 0.019 0.011 0.016 0.004 –0.004 –0.001
α = 1.333 ese 0.334 0.145 0.192 0.102 0.136 0.076 0.097 0.053
bse 1.253 0.136 0.218 0.101 0.136 0.073 0.094 0.053
95% ecp 0.940 0.942 0.954 0.949
τ = 0.50 Bias 0.302 0.036 0.069 0.018 0.022 0.005 –0.009 –0.002
α = 2.0 ese 1.360 0.141 0.455 0.096 0.288 0.068 0.190 0.047
bse 8.808 0.132 0.811 0.093 0.288 0.068 0.196 0.047
95% ecp 0.965 0.951 0.939 0.952
τ = 0.75 Bias 7.136 0.160 0.539 0.030 0.164 0.010 0.014 0.001
α = 4.0 ese 41.24 0.104 3.830 0.073 0.975 0.050 0.635 0.038
bse 44.55 0.089 12.353 0.067 1.590 0.049 0.659 0.036
95% ecp 0.971 0.963 0.960 0.940

5 Application to the motivating example

We apply the proposed method to the sub-cohort of MACS from Williams et al. (2004) to study the association of GBV-C persistence time and HIV survival. MACS consists of gay men who were enrolled between 1984 and 1990 and whose blood samples were obtained every 6 months and tested retrospectively when a test for HIV became available. The sub-cohort includes 271 subjects from MACS who were initially HIV negative when they entered the study but HIV positive during the follow ups. Since the visits were scheduled every 6 months, the seroconversion time is known to be within a six-month window. Seroconversion time is imputed as the midpoint between the last seronegative visit and the first seropositive visit. All 271 subjects were evaluated at 12–18 months after HIV seroconversion for the evidence of GBV-C infection and a subgroup of 138 patients were re-examined 4.5–6 years after HIV seroconversion. The study only included data collected before Jan 1, 1996 to avoid the impact of the use of highly active antiretroviral therapy.

Williams et al. (2004) compared the Kaplan-Meier curves for the survival time of the HIV subjects with and without GBV-C co-infection at disease onset and found no significant difference at level 0.05. Here we consider the association between GBV-C persistence time and HIV survival among people who were co-infected with both HIV and GBV-C at HIV onset. HIV survival is defined as the time from seroconversion to death, and GBV-C persistence time is defined as the time from HIV seroconversion to GBV-C clearance for the subjects with GBV-C positive at HIV onset. Previous clinical studies and lab studies suggest that the re-infection of GBV-C is very rare among people who have already infected with HIV. So we assume that the HIV subjects who were co-infected with GBV-C would not be re-infected once they lose it.

In our analysis, we treat the GBV-C status evaluated at 12–18 months as the baseline GBV-C information to select a subsample of HIV patients who are assumed to be co-infected with GBV-C at HIV onset. The GBV-C status evaluated at the second time after HIV seroconversion presents the current status data for GBV-C persistence time. The Gumbel copula is used for the bivariate distribution of HIV survival and GBV-C persistence times. The bootstrap standard error based on 1,000 resamples with replacement was used to estimate the standard error of the estimated association parameter and to construct the Wald confidence interval. There are 61 subjects who were GBV-C positive at the first visit, and GBV-C status at the late visit were known and evaluated before January 1, 1996. In order to use as many data as possible, we define the current status of GBV-C co-infection for the subjects whose late observations on GBV-C were unavailable before January 1, 1996 as follows: (i) for those whose second GBV-C test were negative and evaluated after January 1, 1996, their GBV-C persistence times were right censored at the first visit (n = 2); (ii) for those whose second GBV-C test were positive and evaluated after January 1, 1996, their GBV-C persistence times were right censored at January 1, 1996 (n = 7); and (iii) for those whose second GBV-C test were missing, their GBV-C persistence times were right censored at the first visit (n = 37). Therefore, we have a total of 107 subjects for analysis. Table 3 presents the results when all the subjects who were GBV-C positive at the first visit are included. The maximum pseudo-likelihood estimate of Kendall's τ is τ^n=0.3685 with an 95% confidence interval being [0.1988, 0.5383] using the asymptotic normality or [0.2114, 0.5533] using the bootstrap method. The result indicates that GBV-C persistence time is moderately associated with increased survival among HIV and GBV-C co-infected individuals.

Table 3.

The analysis of association between HIV survival time and GBV-C persistence time: include all subjects whose GBV-C at early visit are positive (N = 107)

Estimate Bootstrap SE 95% Wald CI 95% Bootstrap CI
α^n
1.5836 0.2037 [1.1843, 1.9829] [1.2043, 2.0598]
τ^n
0.3685 0.0866 [0.1988, 0.5383] [0.2114, 0.5533]

6 Final remarks

This manuscript proposes a method for assessing the association between two random variables which are subject to different censoring schemes: one is right censored and the other is observed as current status data. The asymptotic properties of the estimator of association parameter, including consistency and asymptotic normality, are established under mild technical assumptions. Although the asymptotic variance of the estimator has a complicated form and is difficult to estimate directly, the ordinary bootstrap method provides a practical and efficient way to estimate the standard error.

Our simulation results suggest that the proposed method works well for moderate sample size and has the advantage of allowing for flexibility in the marginal distributions. Moreover, our numerical study shows that the proposed method is quite efficient compared to the full maximum likelihood approach in which the marginal distributions are given. It suggests that the efficiency loss from the pseudo-likelihood approach is not substantial.

Some copula functions, such as the Gumbel copula, are equivalent to the independent copula only when the association parameter takes its value on the boundary of the parameter space. It may result in failure of some regularity conditions and hence the likelihood theory cannot be easily developed which makes the test of the independence of bivariate event times problematic using the copula models. Several nonparametric tests of dependence have been developed for the bivariate censored data (Oakes 1982; Shih and Louis 1996; Hsu and Prentice 1996; Ding and Wang 2004). A nonparametric test procedure to test the dependence between HIV survival and GBV-C persistence time needs to be developed under the hybrid censoring considered in this paper.

A new study testing additional stored samples from MACS cohort is being planned. There will be considerably more power and precision using these additional time points in the analysis. With the new study of more GBV-C screening, GBV-C persistence time is interval censored, the method presented here will be extended to model the bivariate event data with one margin being subject to right censoring and the other being subject to interval censoring case 2. Other applications with time-varying covariates subject to interval censoring case 2 are readily available.

Acknowledgements

The authors wish to thank the Multicenter AIDS Cohort Study (MACS) for providing data. The MACS has centers located at: The Johns Hopkins Bloomberg School of Public Health (Joseph Margolick); Howard Brown Health Center and Northwestern University Medical School (John Phair); University of California, Los Angeles (Roger Detels); University of Pittsburgh (Charles Rinaldo); and Data Analysis Center (Lisa Jacobson). The authors are also thankful to the editors and two anonymous referees. Their insightful comments and suggestions greatly help improve this manuscript from an early version.

Appendix

This section provides proofs for the lemmas and theorems stated in Sect. 3. We use modern empirical process theory to justify our proofs. We denote ∫ fdP by Pf and 1ni=1nf(Xi) by Pnf.

Proof of Lemma 1 Since Fj consists of uniformly bounded monotone functions on the real line, by the Theorem 2.7.5 of ?, for any ε > 0, for j = 1, 2, there exists a set of brackets:

[fj1L,fj1U],[fj2L,fj2U],,[fjNjL,fjNjU],

with Nj ≤ exp (D/ε) and (fjiUfjiLrdP)1r for any 1 ≤ iNj and r > 0, such that for any fjFj and any tj[0,t0j],fjqjL(tj)fj(tj)fjqjU(tj) for some 1 ≤ qjNj.

By (A2), Vα,1(α, f1(t1), f2(t2), δ1, δ2) is continuous. We can then construct a set of brackets as follows: for any i = 1, 2, . . . , N1, s = 1, 2, . . . , N2 and for any tj ∈ [0, t0j], we can find the unique maximum and minimum of Vα,1(α, f1(t1), f2(t2), δ1, δ2) on the product set [f1iL,f1iU]×[f2sL,f2sU]. Let

(f1L,(i,s)(t1),f2L,(i,s)(t2))=argminf1[f1iL,f1iU]f2[f2sL,f2sU]Vα,1(α,f1(t1),f2(t2),δ1,δ2)(f2U,(i,s)(t1),f2U,(i,s)(t2))=argmaxf1[f1iL,f1iU]f2[f2sL,f2sU]Vα,1(α,f1(t1),f2(t2),δ1,δ2)

and let

Vα,1L,(i,s)(t1,t2,δ1,δ2)=Vα,1(α,f1L,(i,s)(t1),f2L,(i,s)(t2),δ1,δ2)Vα,1U,(i,s)(t1,t2,δ1,δ2)=Vα,1(α,f1U,(i,s)(t1),f2U,(i,s)(t2),δ1,δ2).

The class GF is then covered by a set of N1 × N2 brackets:

{[Vα,1L,(i,s)(t1,t2,δ1,δ2),Vα,1U,(i,s)(t1,t2,δ1,δ2)]:i=1,2,,N1,s=1,2,,N2}.

By (A2), Vα,12(α, u, v, δ1, δ2) and Vα,1,2(α, u, v, δ1, δ2) are bounded by some constant D, then Vα,1(α, u, v, δ1, δ2) satisfies the Lipschitz condition with respect to u and v. It follows that:

Vα,1U,(i,s)(t1,t2,δ1,δ2)Vα,1L,(i,s)(t1,t2,δ1,δ2)dP=Vα,1(α,f1U,(i,s)(t1),f2U,(i,s)(t2),δ1,δ2)Vα,1(α,f1L,(i,s)(t1),f2L,(i,s)(t2),δ1,δ2)dP[Df1U,(i,s)(t1)f1L,(i,s)(t1)+Df2U,(i,s)(t2)f2L,(i,s)(t2)]dPD.

This indicates that the preceding N1 × N2 brackets are Dε–brackets for GF. It follows that, for any ε > 0, the bracketing number of class GF associated with L1(P) norm is bounded. By Theorem 2.4.1 of ?, GF is a P-Glivenko-Cantelli class.

Proof of Lemma 2 Based on the similar technique used in the proof of Lemma 1, we can construct a set of N1 × N2 brackets:

{[VαL,(i,s)(t1,t2,δ1,δ2)Vα(α,S1(t1),S2(t2),δ1,δ2),VαU,(i,s)(t1,t2,δ1,δ2)Vα(α,S1(t1),S2(t2),δ1,δ2)]:i=1,2,,N1,s=1,2,,N2},

which covers HF.

By (A2), Vα,1(α, u, v, δ1, δ2) and Vα,2(α, u, v, δ1, δ2) are bounded by some constant D, then Vα(α, u, v, δ1, δ2) satisfies the Lipschitz condition with respect to u and v. Also note that (x + y)2 = x2 + y2 + 2xy ≤ 2x2 + 2y2, it follows that

(VαU,(i,s)(t1,t2,δ1,δ2)VαL,(i,s)(t1,t2,δ1,δ2))2dP=Vα(α,f1U,(i,s)(t1),f2U,(i,s)(t2),δ1,δ2)Vα(α,f1L,(i,s)(t1),f2L,(i,s)(t2),δ1,δ2)2dP[Df1U,(i,s)(t1)f1L,(i,s)(t1)+Df2U,(i,s)(t2)f2L,(i,s)(t2)]2dP2D2f1U,(i,s)(t1)f1L,(i,s)(t1)2dP+2D2f2U,(i,s)(t2)f2L,(i,s)(t2)2dPD2.

This indicates that the bracketing number of HF associated with L2(P) norm, denoted by N[](,HF,L2(P)), is bounded by N1 × N2. It follows that log(N[](,HF,L2(P)))log(N1×N2)D for some constant D. Hence,

01logN[](,HF,L2(P))d01D12d<.

By Theorem 19.5 of van der Vaart and Wellner (1996, p. 270), HF is a P-Donsker Class.

Proof of Theorem 1 Let

Ln(α,S^1,S^2;X)=1ni=1nl(α,S^1(t1i),S^2(t2i),δ1i,δ2i)Ln(α,S1,S2;X)=1ni=1nl(α,S1(t1i),S2(t2i),δ1i,δ2i),

where l(α, Ŝ1(t1i), Ŝ2(t2i) is defined in (3). First we show that Ln(α,S^1,S^2;X)pEα0l(α,S1(T1),S2(T2),Δ1,Δ2) for any α ∈ A.

By Taylor series expansion, we have

Ln(α,S^1,S^2;X)=Ln(α,S1,S2;X)+1ni=1n(S^1(t1i)S1(t1i))Vα,1(α,S~1(t1i),S^2(t2i),δ1i,δ2i)+1ni=1n(S^2(t2i)S2(t2i))Vα,2(α,S^1(t1i),S~2(t2i),δ1i,δ2i)

where supt1[0,t01]S~1(t1)S1(t1)supt1[0,t01]S^1(t1)S1(t1)p0 (Fleming and Harrington 1991) and supt2[0,t02]S~2(t2)S2(t2)supt2[0,t02]S^2(t2)S2(t2)p0 (Groeneboom and Wellner 1992), respectively.

By the Weak Law of Large Number Theorem,

Ln(α,S1,S2;X)pEα0l(α,S1(T1),S2(T2),Δ1,Δ2).

Note that

1ni=1n(S^1(t1i)S1(t1i))Vα,1(α,S~1(t1i),S^2(t2i),δ1i,δ2i)supt1[0,t01]S^1(t1)S1(t1)PnVα,1(α,S~1(T1),S^2(T2),Δ1,Δ2)

Denote GF={Vα,1(α,f1(t1),f2(t2),δ1,δ2);fjFj,j=1,2}. Since GF is a P-Glivenko-Cantelli class by Lemma 1, a straightforward algebra yields that the ε-bracketing number of GF is the same as the ε-bracketing number of GF which results in GF being a P-Glivenko-Cantelli class as well. Hence

PnVα,1(α,S~1(T1),S^2(T2),Δ1,Δ2)=PVα,1(α,S~1(T1),S^2(T2),Δ1,Δ2)+op(1)=PVα,1(α,S1(T1),S2(T2),Δ1,Δ2)+op(1),

due to the uniform consistency of 1(·) and Ŝ2(·), the continuous mapping theorem, assumption (A2), and the dominated convergence theorem. This implies that

1ni=1n(S^1(t1i)S1(t1i))Vα,1(α,S~1(t1i),S^2(t2i),δ1i,δ2i)p0.

Similar argument leads that

1ni=1n(S^2(t2i)S2(t2i))Vα,2(α,S^1(t1i),S~2(t2i),δ1i,δ2i)p0.

This concludes Ln(α,S^1,S^2;X)pEα0l(α,S1(T1),S2(T2),Δ1,Δ2). Now, αA, using Jensen's inequality, it follows that

Ln(α,S^1,S^2;X)Ln(α0,S^1,S^2;X)pEα0l(α,S1(T1),S2(T2),Δ1,Δ2)Eα0l(α0,S1(T1),S2(T2),Δ1,Δ2)=Eα0logh(α,T1,T2,Δ1,Δ2)h(α0,T1,T2,Δ1,Δ2)<logEα0h(α,T1,T2,Δ1,Δ2)h(α0,T1,T2,Δ1,Δ2)=0.

Due to the convergence demonstrated above, , δ > 0, for which (α0ε, α0 + ε) ∈ A, we may find an integer N = N (ε, δ), such that, if n > N, for α = α0 ± ε,

P(Ln(α,S^1,S^2;X)<Ln(α0,S^1,S^2;X))>1δ.

Thus for n > N,

P(Ln(α,S^1,S^2;X)has a local maximumα^n(α0,α0+))>12δ, because of (A1). This immediately shows that the sequence of random variables α^n converge in probability to α0 as n → ∞.

Proof of Theorem 2 Under (A1), Taylor expansion of the pseudo score function gives

0=PnVα(α^n,S^1(T1),S^2(T2),Δ1,Δ2)=PnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2)+(α^nα0)PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)+Op(α^nα02),

then we get

n(α^nα0)=nPnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2)PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)Op(α^nα0).

First, we show that

PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)pW(α0,S1,S2),

where

W(α0,S1,S2)=PVα2(α0,S1(T1),S2(T1),Δ1,Δ2)=P[Vα(α0,S1(T1),S2(T2),Δ1,Δ2)]2.

We can rewrite PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)=PnVα2(α0,S1(T1),S2(T2),Δ1,Δ2)+Rn. By the uniform consistency of Ŝ1 and Ŝ2 and the fact that Vα2 (α0, S1(t1), S2(t2), δ1, δ2) satisfies the Lipschitz condition due to (A2), it follows that

PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)=PnVα2(α0,S1(T1),S2(T2),Δ1,Δ2)+op(1).

This results in

PnVα2(α0,S^1(T1),S^2(T2),Δ1,Δ2)pPVα2(α0,S1(T1),S2(T2),Δ1,Δ2)

by the Weak Law of Large Number Theorem.

Second, we derive the asymptotic distribution of nPnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2). Note that

PnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2)=(PnP)(Vα(α0,S^1(T1),S^2(T2),Δ1,Δ2)Vα(α0,S1(T1),S2(T2),Δ1,Δ2))+PnVα(α0,S1(T1),S2(T2),Δ1,Δ2)+P(Vα(α0,S^1(T1),S^2(T2),Δ1,Δ2)Vα(α0,S1(T1),S2(T2),Δ1,Δ2))=u1n+u2n+u3n.

Lemma 2 indicates that under (A1) and (A2), HF is a P-Donsker class. Furthermore, since sup0tjt0jS^j(tj)Sj(tj)p0,j=1,2, by the Dominated Convergence Theorem,

(S^j(tj)Sj(tj))2dP(t1,t2,δ1,δ2)p0,j=1,2.

Therefore, nu1n=op(1) by Lemma 19.24 of van der Vaart and Wellner (1996).

Note that u2n is a sum of independent and identically distributed quantities, where each quantity has mean

Vα(α0,S1(t1),S2(t2),δ1,δ2)dP(t1,t2,δ1,δ2)=0

and variance

[Vα(α0,S1(t1),S2(t2),δ1,δ2)]2dP(t1,t2,δ1,δ2)=W(α0,S1,S2).

By the Central Limit Theorem, nu2n converges in distribution to a normal random variable with mean 0 and variance – W (α0, S1, S2).

Applying Von Mises Expansion (von Mises 1947) on u3n around S1, S2, we get

u3n=d0t01IC1(t1)d(S^1S1)(t1)+0t02IC2(t2)d(S^2S2)(t2), (4)

where ICj (t), j = 1, 2 are the influence curves of the functional PVα(α0, S1(T1), S2(T2), Δ1, Δ2) which are defined by

IC1(t1)=0t10t02Vα,1(α0,S1(τ1),S2(τ2),δ1,δ2)dP(τ1,τ2,δ1,δ2)=0t10t02Mα,1(α0,S1(τ1),S2(τ2))f(τ1,τ2)dτ1dτ2

and

IC2(t2)=0t20t01Vα,2(α0,S1(τ1),S2(τ2),δ1,δ2)dP(τ1,τ2,δ1,δ2)=0t20t01Mα,2(α0,S1(τ1),S2(τ2))f(τ1,τ2)dτ1dτ2, (5)

respectively. Here

Mα,1(α0,S1(τ1),S2(τ2))=E{Vα,1(α0,S1(T1),S2(T2),Δ1,Δ2)T1=τ1,T2=τ2}Mα,2(α0,S1(τ1),S2(τ2))=E{Vα,2(α0,S1(T1),S2(T2),Δ1,Δ2)T1=τ1,T2=τ2}.

Using the martingale theory for counting process, Pepe (1991) showed that, for t ∈ [0, t01], (Ŝ1 (t1) – S1 (t1)) is asymptotically equivalent to a sum of n i.i.d. random variables iI10(T1i,Δ1i)(t1)n. It follows that

0t01IC1(t1)d(S^1S1)(t1)=1ni=1nI1(T1i,Δ1i;α0)+op(1), (6)

where

I1(T1i,Δ1i;α0)=0t010t02Mα,1(α0,S1(τ1),S2(τ2))f(τ1,τ2)I10(T1i,Δ1i)(τ1)dτ1dτ2

and I10 is a martingale given by

I10(T1,Δ1)(t1)=S1(t1){0t11P(T1u)dN1(u)0t1I[T1u]P(T1u)dΛ1(u)},

in which N1(u) is defined as I[T1u, Δ1 = 1] and Λ1 is the cumulative hazard function of T10.

On the other hand, although (Ŝ2S2)(t2) can not be written as sum of i.i.d random quantities, a smooth functional of the nonparametric maximum likelihood estimator Ŝ2 can still be shown asymptotically normal (Huang and Wellner 1995). Using this property and the regularity conditions (A3)–(A6), Wang and Ding (2000) showed that

0t02IC2(t2)d(S^2S2)(t2)=Pnl~(2,Δ2;S2,G2,ψ2)+op(1) (7)

with l~(T2,Δ2;S2,G2,ψ2)=[Δ2(1S2(T2))]ψ2(T2)g2(T2)I[g2(T2)>0] and thus n0t02IC2(t2)d(S^2S2)(t2) converges in distribution to a normal random variable with mean 0.

In summary, we obtain that,

PnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2)=Pn[Vα(α0,S1(T1),S2(T2),Δ1,Δ2)+I1(T1,Δ1;α0)l~(T2,Δ2;S2,G2,ψ2)]+op(n12)=PnQ(T1,T2,Δ1,Δ2;α0,S1,S2)+op(n12).

Therefore, nPnVα(α0,S^1(T1),S^2(T2),Δ1,Δ2) is asymptotically normal with mean zero and variance Var(Q(T1, T2, Δ1, Δ2; α0, S1, S2)). Hence,

n(α^nα0)dN(0,σ2),

where

σ2=Var(Q(T1,T2,Δ1,Δ2;α0,S1,S2))W2(α0,S1,S2).

Contributor Information

Suhong Zhang, Division of Biostatistics, Edwards Lifesciences, One Edwards Way, Irvine, CA 92612, USA suhong.zhang@edwards.com.

Ying Zhang, Department of Biostatistics, University of Iowa, C22 GH, 200 Hawkins Drive, Iowa City, IA 52242, USA ying-j-zhang@uiowa.edu.

Kathryn Chaloner, Department of Biostatistics, University of Iowa, C22 GH, 200 Hawkins Drive, Iowa City, IA 52242, USA kathryn-chaloner@uiowa.edu.

Jack T. Stapleton, Department of Internal Medicine, University of Iowa and Iowa City VA Medical Center, SW54-15 GH, 200 Hawkins Drive, Iowa City, IA 52242, USA jack-stapleton@uiowa.edu

References

  1. Birk M, Lindback S, Lidman C. No influence of GB virus C replication on the prognosis in a cohort of HIV-1-infected patients. AIDS. 2002;16:2482–2485. doi: 10.1097/00002030-200212060-00017. [DOI] [PubMed] [Google Scholar]
  2. Ding AA, Wang W. Testing independence for bivariate current status data. J Am Stat Assoc. 2004;99:145–155. [Google Scholar]
  3. Fleming TR, Harrington DP. Counting process and survival analysis. John wiley & Sons; New York: 1991. [Google Scholar]
  4. Groeneboom P, Wellner JA. Information bounds and nonparametric maximum likelihood estimation. Birkhauser; Boston: 1992. [Google Scholar]
  5. Hougaard P. Fitting a multivariate failure time distribution. IEEE Trans Reliab. 1989;38:444–448. [Google Scholar]
  6. Hsu L, Prentice RL. A generalisation of the mantel-haenszel test to bivariate failure time data. Biometrika. 1996;4:905–911. [Google Scholar]
  7. Huang J, Wellner JA. Asymptotic normality of the npmle of linear functionals for interval censored data, case 1. Stat Neerl. 1995;49:153–163. [Google Scholar]
  8. Kalbfleisch JD, Prentice RL. The statistical analysis of failure time data. 2nd edn. Wiley-Interscience; New York: 2002. [Google Scholar]
  9. Liang KE, Self SG, Bandeen-Rocche K, Zeger S. Some recent developments for regression analysis of multivariate failure time data. Lifetime Data Anal. 1995;1:403–415. doi: 10.1007/BF00985452. [DOI] [PubMed] [Google Scholar]
  10. Nelsen RB. An introduction to copulas. 2nd edn. Springer-Verlag; New York: 2006. [Google Scholar]
  11. Oakes D. A concordance test for independence in the presence of censoring. Biometrics. 1982;38:451–455. [PubMed] [Google Scholar]
  12. Oakes D. Bivariate survival models induced by frailties. J Am Stat Assoc. 1989;84:487–493. [Google Scholar]
  13. Oakes D. Survival analysis. J Am Stat Assoc. 2000;95:282–285. [Google Scholar]
  14. Pepe MS. Inference for events with dependent risks in multiple endpoint studies. J Am Stat Assoc. 1991;86:770–778. [Google Scholar]
  15. Shih JH, Louis T. Inference on the association parameter in copula models for bivariate survival data. Biometrics. 1995;51:1384–1399. [PubMed] [Google Scholar]
  16. Shih JH, Louis TA. Tests of independence for bivariate survival data. Biometrics. 1996;4:1440–1449. [PubMed] [Google Scholar]
  17. Tillmann H, Heiken H, Knapik-Botor A, Heringlake S, Ockenga J, et al. Infection with GB virus C and reduced mortality among HIV-infected patients. N Engl J Med. 2001;345:715–724. doi: 10.1056/NEJMoa010398. [DOI] [PubMed] [Google Scholar]
  18. Toyoda H, Fukuda Y, et al. Effect of GB virus C/hepatitis G virus coinfection on the course of HIV infection in hemophilia patients in Japan. J Acquir Immune Defic Syndr Hum Retrovirol. 1998;17:209–213. doi: 10.1097/00042560-199803010-00004. [DOI] [PubMed] [Google Scholar]
  19. van der Vaart AW. Asymptotic statistics. Cambridge Univ. Press; Cambridge: 1998. [Google Scholar]
  20. van der Vaart AW, van der Wellner JA. Weak convergence and empirical processes with application to statistics. Springer-Verlag; New York: 1996. [Google Scholar]
  21. von Mises R. On the asymptotic distribution of differentiable statistical functions. Ann Math Statist. 1947;18:309–348. [Google Scholar]
  22. Wang W, Ding AA. On assessing the association for bivariate current status data. Biometrika. 2000;87:879–893. [Google Scholar]
  23. Williams C, Klinzman D, Yamashita T, Xiang J, et al. Persistent GB virus C infection and survival in HIV-infected men. N Engl J Med. 2004;350:981–990. doi: 10.1056/NEJMoa030107. [DOI] [PubMed] [Google Scholar]
  24. Xiang J, Wunschmann W, Diekema D, Klinzman D, Patrick K, et al. Effect of coinfection with GB virus C on survival among patients with HIV infection. N Engl J Med. 2001;345:707–714. doi: 10.1056/NEJMoa003364. [DOI] [PubMed] [Google Scholar]
  25. Zhang W, Chaloner K, Tillmann HS, Williams CF, Stapleton JT. Effect of early and late GBV-C viremia on survival of HIV-infected individuals: a meta-analysis. HIV Med. 2006;7:173–180. doi: 10.1111/j.1468-1293.2006.00366.x. [DOI] [PubMed] [Google Scholar]

RESOURCES