Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2017 May 15.
Published in final edited form as: Stat Interface. 2016;9(3):333–341. doi: 10.4310/SII.2016.v9.n3.a7

Semiparametric Random Effects Models for Longitudinal Data with Informative Observation Times

Yang Li 1,, Yanqing Sun 2
PMCID: PMC5431605  NIHMSID: NIHMS851581  PMID: 28515829

Abstract

Longitudinal data frequently arise in many fields such as medical follow-up studies focusing on specific longitudinal responses. In such situations, the responses are recorded only at discrete observation times. Most existing approaches for longitudinal data analysis assume that the observation or follow-up times are independent of the underlying response process, either completely or given some known covariates. We present a joint analysis approach in which possible correlations among the responses, observation and follow-up times can be characterized by time-dependent random effects. Estimating equations are developed for parameter estimation and the resulting estimates are shown to be consistent and asymptotically normal. A simulation study is conducted to assess the finite sample performance of the approach and the method is applied to data arising from a skin cancer study.

Keywords and phrases: estimating equations, informative censoring, informative observation process, joint analysis approach, longitudinal data

1. INTRODUCTION

Longitudinal data arise in many fields such as medical follow-up studies that focus on longitudinal responses. In such situations, each study subject is observed only at finite discrete times rather than continuously. Therefore, the responses are known only at a set of observation times but missing otherwise. The resulting data are usually incomplete and unbalanced among individuals.

Analysis of longitudinal data concerns two processes: one is the underlying response process, which is usually of practical interest but not continuously observable. The other refers to the observation process, which determines the discrete observation times. Many authors have considered the analysis of longitudinal data, for example, Diggle et al. (1994) who presented a relatively comprehensive review about the commonly considered models and estimation methods. Lin and Ying (2001), Welsh et al. (2002), Wellner and Zhang (2007) and Sun (2010) developed some semiparametric and nonparametric procedures for regression analysis. These approaches all assume that the two processes mentioned above are independent, either completely or conditional on some known covariates. To relax this assumption, Sun et al. (2007b), Zhao and Tong (2011) and Zhao et al. (2013b) modeled the possible correlations by time-independent random effects. However, these methods assume follow-up times to be independent from both the response and the observation processes given covariates.

In many situations, the underlying response process, the observation and follow-up times may be correlated. For example, both observation times and responses may depend on the stage of disease progression, which can also often determine the follow-up time. Lipsitz et al. (2002) considered general linear models for longitudinal data where the responses were assumed to have a multivariate Gaussian distribution. Sun et al. (2007), He et al. (2009) and Sun et al. (2012) proposed joint model based approaches; however, it is assumed that the shared random effects are fixed over time or follow some specific distributions, and the covariates are either multiplicative or additive in their effects to the response process. Without such specific distribution assumption, Sun et al. (2005) and Zhao et al. (2013) considered marginal model based methods; however, the models indicate that when the observation process is common for everyone, people with the same covariates are expected to have the same responses throughout the study. It is apparent that such assumptions may not be realistic in many applications.

We present a joint analysis approach for longitudinal data by which the possible correlations can be characterized by time-dependent random effects with arbitrary distributions. For the response process, a class of semiparametric transformation models are considered. Estimating equations are developed for parameter estimation and the resulting estimators are shown to be consistent and asymptotically normal. The remainder of this paper is organized as follows. We introduce notation and present the relevant models in Section 2. Section 3 presents the estimation procedure and establishes asymptotic properties of the proposed estimators. In Section 4, we demonstrate a model-checking technique and an extensive simulation study is presented in Section 5 to evaluate finite sample properties of the estimation procedure. An illustrative example is given in Section 6 and some discussion and remarks are provided in Section 7.

2. NOTATION AND MODELS

Consider a longitudinal study in which subjects are observed only at discrete times. For subject i (i = 1, …, n), let Yi(t) denote the response process and let Ni(t) be the observation process which gives the cumulative number of observations at time t. In practice, one often observes Ñi(t) = Ni(tCi) where ab = min(a, b) and Ci denotes a possible censoring or follow-up time. Let {Ti,1, · · ·, Ti,mi} be the discrete times when Yi(t) is observed and let Zi(t) be a p-dimensional vector of covariates, assumed to be continuously traceable in the study. In the following, we present a joint modeling approach and model the possible correlation between Yi(t), Ni(t) and Ci through an unobserved random process bi(t) = (b1i(t), b2i(t), b3i(t))′. Define ℬit = {bi(s), st} and 𝒵it = {Zi(s), st}. We assume that the bi(t)’s are independent and identically distributed with b1i(t) > 0 and b2i(t) > 0, ℬit is independent of 𝒵it, and given 𝒵it and ℬit, Ci, Ni(t) and Yi(t) are mutually independent. Also we assume that the mean function of Yi(t) can be postulated by the following semiparametric transformation model

E{Yi(t)Zi(t),bi(t)}=g{μ0(t)eθZi(t)}b1i(t), (1)

where g(·) is a known twice continuously differentiable and strictly increasing link function, θ is a vector of unknown regression parameters and μ0(t) denotes an unspecified smooth function of t. We assume that E{b1i(t)|d Ñi(t) = 1,𝒵it} = 1 for identifiability. In particular, when g(x) = x, μ0(t) represents the baseline mean function that is estimable at {Ti,1, · · ·, Ti,mi}.

The observation process Ni(t) is assumed to follow the marginal proportional rates model given by

E{dNi(t)Zi(t),bi(t)}=exp{γZi(t)}b2i(t)dΛ0(t), (2)

where E{b2i(t)} = 1, γ is a vector of unknown parameters and dΛ0(t) is an unknown baseline rate function. It can be seen that both of the above models can be viewed as natural generalizations of the transformation model and proportional rates model studied in Li et al. (2010), Zhao et al. (2011) and Zhao et al. (2013) among others. Compared with the existing models, the proposed models are relatively flexible in handling the possible dependence since neither the form nor the distribution of bi(t) needs to be specified. By taking different forms of g(·) and bi(t), model (1) allows for various types of dependence for the mean function of Yi(t) on Ni(t) and Zi(t). In particular, when either b1i(t) or b2i(t) is unity or independent of the other one, the two processes Yi(t) and Ni(t) are independent given 𝒵it. Therefore, the estimation procedure proposed next also applies to data with noninformative observation times as special cases.

For the follow-up or censoring time Ci, we consider the following additive hazards model

λi(tZi(t),bi(t))=λ0(t)+ξZi(t)+b3i(t), (3)

where E{b3i(t)} = 0, λ0(t) is an unknown baseline hazard function and ξ is an unknown vector of regression parameters. The random effects b1i(t), b2i(t) and b3i(t) characterize possible correlations between Ci and Yi(t), Ni(t), for which b3i(t) = 0 implies noninformative censoring. The same model has also been studied in Kalbfleisch and Prentice (2002), Lin et al. (1998), Zhang et al. (2005) and Sun et al. (2013) among others. In the following, we study the joint analysis of the proposed models with the focus on estimation of regression parameters θ along with γ and ξ.

3. ESTIMATION PROCEDURE

In this section, we present an estimation procedure for θ which is usually of primary interest. To this end, first note that Ñi(t) jumps by one at time t if and only if Cit and dNi(t) = 1. Based on the conditional independence assumption between Ci, Ni(t) and Yi(t) given 𝒵it and ℬit, we have, under (2)

E{dNi(t)Zit}=E[E{I(tCi)dNi(t)Zit,it}|Zit]=E[E{I(tCi)Zit,it}E{dNi(t)Zit,it}|Zit]=E{I(tCi)b2i(t)Zit}exp{γZi(t)}dΛ0(t). (4)

By the property of double expectation and model (3), the first term in (4) equals

E{I(tCi)b2i(t)Zit}=E{exp{-Λ0(t)-Bi(t)-ξZi(t)}b2i(t)|Zit},

where Λ0(t)=0tλ0(s)ds,Bi(t)=0tb3i(s)ds and Zi(t)=0tZi(s)ds. Hence,

E{dNi(t)Zit}=exp{ηXi(t)}dΛ1(t), (5)

where η = (γ, ξ′)′, Xi(t)=(Zi(t),-Zi(t)) and dΛ1(t)=exp{-Λ0(t)}E[b2i(t)exp{-Bi(t)}]dΛ0(t).

Let τ be a known constant representing the length of the study. Define dMi(t;η)=dNi(t)-eηXi(t)dΛ1(t) and dMi(t)=dMi(t;η0), where η0 denotes the true value of η. It is straightforward to show that Mi(t) is a mean-zero stochastic process. It follows that η and Λ1(t) can be estimated by η̂ and Λ^1(t;η^), respectively, by solving the following two estimating equations

Uη(η)=i=1n0τ{Xi(t)-X¯(t;η)}dNi(t)=0, (6)

and

i=1n[dNi(t)-eηXi(t)dΛ1(t)]=0. (7)

where *(t; η) = S(1)(t; η)/S(0)(t; η) and S(k)(t;η)=n-1i=1neηXi(t)Xi(t)k for k = 0, 1 and 2. Here and throughout a⊗0 = 1, a⊗1 = a and a⊗2 = aa′. Define Λ^1(t)=Λ^1(t;η^), *(t) = limn→∞ *(t; η0) and s(k)(t) = limn→∞S(k)(t; η0).

For the estimation of θ, consider

E{Yi(t)dNi(t)Zit}=E{Yi(t)I(dNi(t)=1)Zit}=E{Yi(t)I(dNi(t)=1)Zit}E{I(dNi(t)=1)Zit}E{dNi(t)Zit}

by the definition of d Ñi(t) and simple manipulation. From the conditional independence assumption between Ci, Ni(t) and Yi(t) given 𝒵it and ℬit, the last equality equals

E{Yi(t)dNi(t)Zit}=E[E{Yi(t)I(dNi(t)=1)Zit,it}Zit]E{I(dNi(t)=1)Zit}E{dNi(t)Zit}=g{μ0(t)eθZi(t)}E{b1i(t)I(dNi(t)=1)Zit}E{I(dNi(t)=1)Zit}×E{dNi(t)Zit}=g{μ0(t)eθZi(t)}E{b1i(t)dNi(t)=1,Zit}×E{dNi(t)Zit}=g{μ0(t)eθZi(t)}E{dNi(t)Zit}.

under models (1). Combining (5), it follows that

E{Yi(t)dNi(t)Zit}=eηXi(t)g{μ0(t)eθZi(t)}dΛ1(t). (8)

We define

dMi(t;θ,η)=Yi(t)dNi(t)-eηXi(t)g{μ0(t)eθZi(t)}dΛ1(t)

and dMi(t) = dMi(t; θ0, η0), where θ0 denotes the true value of θ. Then Mi(t) is a mean-zero stochastic process, which naturally suggests the following estimating equations to estimate θ and μ0(t):

i=1n[Yi(t)dNi(t)-eη^Xi(t)g{μ0(t)eθZi(t)}dΛ^1(t)]=0,0tτ, (9)

and

i=1n0τW(t)Zi(t)×[Yi(t)dNi(t)-eη^Xi(t)g{μ0(t)eθZi(t)}dΛ^1(t)]=0, (10)

where W(t) is a possibly data-dependent weight function. We denote the estimates of θ and μ0(t) by θ̂ and μ̂0(t; θ̂, η̂), respectively. Define μ̂0(t) = μ̂0(t; θ̂, η̂).

In general, neither θ̂ nor μ̂0(t) have closed forms and some iterative algorithms may be necessary to solve (9) and (10). For some special cases, μ̂0(t) can be written explicitly. For example, when g(x) = log(x), it can be shown that

μ^0(t)=exp{i=1nYi(t)dNi(t)i=1neη^Xi(t)dΛ^1(t)-θ^Z¯(t;η^)}

and

θ^={i=1n0τW(t){Zi(t)-Z¯(t;η^)}Zi(t)eη^Xi(t)dΛ^1(t)}-1×i=1n0τW(t){Zi(t)-Z¯(t;η^)}Yi(t)dNi(t).

where Z¯(t;η^)=i=1nZi(t)eη^Xi(t)i=1neη^Xi(t).

To establish the asymptotic properties of θ̂, we define

M^i(t)=Ni(t)-0teη^Xi(s)dΛ^1(s),M^i(t)=0tYi(s)dNi(s)-0teη^Xi(s)g{μ^0(s)eθ^Zi(s)}dΛ^1(s),E^Z(t;θ^,η^)=i=1nZi(t)g.{μ^0(t)eθ^Zi(t)}eθ^Zi(t)+η^Xi(t)i=1ng.{μ^0(t)eθ^Zi(t)}eθ^Zi(t)+η^Xi(t)ez(t)=limnE^Z(t;θ0,η0)andE^Z(t)=E^Z(t;θ^,η^).

The following theorem establishes the consistency and asymptotic normality of θ̂ and η̂.

Theorem 1

Assume that the conditions (C1)–(C5) given in the Appendix hold. Then θ̂ and η̂ are consistent estimators of θ0 and η0, respectively. n1/2(θ̂–θ0) and n1/2(η̂–η0) converge weakly to mean-zero normal distributions with covariance matrices that can be consistently estimated by ^θ=A^θ-1^A^θ-1 and ^η=Ω^η-1Ψ^Ω^η-1, respectively, where ^=n-1i=1n(ξ^1i-ξ^2i-ξ^3i)2,Ψ^=n-1i=1nζ^i2,

ξ^1i=0τW(t)(Zi(t)-E^Z(t))dM^i(t)ξ^2i=0τW(t)D^(t;θ^,η^)S(0)(t;η^)dM^i(t),ξ^3i=0τA^ηΩ^η-1(Xi(t)-X¯(t;η^))dM^i(t),ζ^i=0τ(Xi(t)-X¯(t;η^))dM^i(t),A^θ=1ni=1n0τW(t)g.{μ^0(t)eθ^Zi(t)}×{Zi(t)-E^Z(t)}2eθ^Zi(t)+η^Xi(t)μ^0(t)dΛ^1(t),D^(t;θ^,η^)=1ni=1n{Zi(t)-E^Z(t)}g{μ^0(t)eθ^Zi(t)}eη^Xi(t),A^η=1ni=1n0τW(t)g{μ^0(t)eθ^Zi(t)}eη^Xi(t)×{Zi(t)-E^Z(t)}{Xi(t)-X¯(t;η^)}dΛ^1(t),Ω^η=1ni=1n0τ{Xi(t)-X¯(t;η^)}2eη^Xi(t)dΛ^1(t).

The proof of the theorem above is sketched in Appendix A.

4. MODEL CHECKING

As mentioned above, a main advantage of the proposed methodology is that it is applicable to a class of correlated models through the link function g(·) and random effects bi(t). On the other hand, one may question how to choose an appropriate form of g(·) for the response process. To answer this question, one may develop some model selection procedure and choose an optimal g(·) among several candidate models. However, such a strategy can be very difficult for longitudinal data because of their incompleteness. To access the adequacy of the proposed models with a given link function g(·), one can develop an omnibus goodness-of-fit test based on the cumulative summation of the residual process (Lin et al., 1993; Lin et al., 2000; Li et al., 2010; Zhao et al., 2013) as follows

(t,x)=n-1/2i=1n0tI(Zi(s)z)dM^i(s),

where {Zi(u) ≤ z} represents that each component of Zi(u) is no greater than the corresponding component of z. In general, the distribution of ℱ(t, x) is unknown or very difficult to obtain. Under the proposed models, ℱ(t, x) is expected to flunctuate randomly around 0. In Appendix B, it is shown that the null distribution of ℱ(t, x) can be approximated by a mean-zero Gaussian distribution

^(t,z)=n-1/2i=1n{u^1i(t,z)-u^2i(t,z)-V^η(t,z)Ω^η-1ζ^i-V^θ(t,z)A^θ-1(ξ^1i-ξ^2i-ξ^3i)}ei, (11)

where e1, e2, …, en are independent standard normal variables independent of the observed data,

u^1i(t,z)=0t{I(Zi(s)z)-E^I(s,z;θ^,η^)}dM^i(s),u^2i(t,z)=0tΓ^(s;θ^,η^)S(0)(s;η^)dM^i(s),Γ^(t;θ^,η^)=n-1i=1n{I(Zi(t)z)-E^I(t,z;θ^,η^)}×g{μ^0(t)eθ^Zi(t)}eη^Xi(t),V^η(t,z)=n-1i=1n0tg{μ^0(s)eθ^Zi(s)}eη^Xi(s){I(Zi(s)z)-E^I(s,z;θ^,η^)}×{Xi(s)-X¯(s;η^)}dΛ^1(s),V^θ(t,z)=n-1i=1n0tg.{μ^0(s)eθ^Zi(s)}I(Zi(s)z)×{Zi(s)-E^Z(s)}eθ^Zi(s)+η^Xi(s)μ^0(s)dΛ^1(s),E^I(t,z;θ^,η^)=i=1nI(Zi(t)z)g.{μ^0(t)eθ^Zi(t)}eθ^Zi(t)+η^Xi(t)i=1ng.{μ^0(t)eθ^Zi(t)}eθ^Zi(t)+η^Xi(t)eI(t,z)=limnEI(t,z;θ0,η0)

and ζ̂i, ξ̂1i, ξ̂2i, ξ̂3i are the same as defined in the previous section. Therefore for a given set of data, one can obtain a large number of realizations from ℱ̂(t, z) by repeatedly generating standard normal random samples {e1, e2, …, en}. A formal goodness-of-fit test can be performed with the corresponding p-value being calculated by comparing sup0≤tτ,z|ℱ(t, z)| to a large number of realizations from sup0≤tτ,z| ℱ̂(t, z)|.

5. A SIMULATION STUDY

In this section, we present results obtained from an extensive simulation study conducted to assess the finite sample behavior of the estimation procedure proposed in the previous sections. In the study, the covariate Zi was assumed to be a Bernoulli random variable with the probability of success being 0.5. Given Zi and some unobserved random effects bi(t) = (b1i(t), b2i(t), b3i(t))′, the hazard function of the censoring time Ci was assumed to have the form

λi(tZi,bi(t))=λ0-ξZi+b3i(t), (12)

with the length of study τ being 1. The number of observations Ni(t) was assumed to follow a Poisson process on (0, Ci) with the rate function

E{dNi(t)Zi,bi(t)}=exp{γZi}b2i(t)dΛ0(t). (13)

In practice, the exact time of Ci may not be observable and d Ñi(t) is observed instead of dNi(t); thus we considered E{Ñi(t)|Zi,it} for the observation times. From (12) and (13),

E{dNi(t)Zi,it}=exp{γZi+ξZit}dΛ1(t),

where dΛ1(t)=exp{-λ0t-Bi(t)}b2i(t)dΛ0(t). Given Zi and bi(t), Ñi(t) was assumed to follow a nonhomogeneous Poisson process and the total number of observation times mi was generated with mean E{mi} = E{Ñi(τ )|Zi, }. Then the observation times {Ti,1, …, Ti,mi} were taken as mi order statistics from the density function

fN(t)=exp{γZi+ξZit}dΛ1(t)0τexp{γZi+ξZit}dΛ1(t).

To generate Yi(Ti,j) at each observation time Ti,j, we considered

E{Yi(Ti,j)Zi,bi(t)}=g{μ0(t)eθZi}b1i(t),

and obtained Yi(Ti,j) by first generating Yi(Ti,j) from a Poisson distribution with the mean function of Yi(t) being equal to g{μ0(t)eθZi}b1i(t)E{I(tCi)|Zi,it}, and then taking Yi(Ti,j)=Yi(Ti,j)E{I(Ti,jCi)Zi,it}. The results given below are based on the sample sizes of 100 and 200 with 1, 000 replications and W(t) = 1.

We took λ0 = 2, dΛ0(t)=5t(e0.5-e-0.5)(et-e-t)dt,b1i=2evie-1/e,b2i(t)=2teui+vit(e0.5-e-0.5)(et-e-t) and b3i = vi with ui and vi being random numbers generated from uniform distributions over (−0.5, 0.5) and (−1, 1), respectively. Table 1 shows the estimation results for θ based on the simulated data with the link function g(x) = log(x), μ0(t)} = e2t, and the true values of (γ, ξ) being equal to (0, 0), (0, 0.2), (0.5, 0), (0.5, 0.2). The table includes the estimated bias given by the average of the proposed estimators θ̂ minus the true value θ0, the average of the estimated standard errors (SEE), the empirical sampling standard error (SSE) and the 95% empirical coverage probability (CP). It can be seen that the proposed approach seems to perform well. Specifically, the proposed estimate seems to be unbiased and the estimated standard errors agree well with the empirical ones. Also as expected, the CP’s are close to their nominal levels and the standard errors become smaller when sample sizes increase.

Table 1.

Estimation results for θ with the link function g(x) = log(x).

θ0 n = 100 n = 200

0 0.2 0.5 0 0.2 0.5
(γ0, ξ0) = (0, 0)
Bias −0.004 −0.006 −0.018 −0.003 0.006 −0.008
SEE 0.186 0.199 0.218 0.131 0.140 0.154
SSE 0.193 0.208 0.220 0.129 0.149 0.149
CP 0.944 0.935 0.943 0.958 0.943 0.962
(γ0, ξ0) = (0, 0.2)
Bias 0.033 0.029 0.024 0.021 0.028 0.024
SEE 0.180 0.192 0.211 0.129 0.137 0.152
SSE 0.187 0.206 0.214 0.133 0.137 0.155
CP 0.939 0.929 0.953 0.942 0.947 0.948
(γ0, ξ0) = (0.5, 0)
Bias 0.005 0.002 0.000 −0.005 −0.001 −0.008
SEE 0.169 0.181 0.199 0.121 0.129 0.142
SSE 0.174 0.185 0.205 0.124 0.134 0.145
CP 0.943 0.950 0.946 0.942 0.943 0.949
(γ0, ξ0) = (0.5, 0.2)
Bias 0.017 0.033 0.012 0.020 0.024 0.018
SEE 0.169 0.177 0.196 0.120 0.127 0.139
SSE 0.171 0.183 0.199 0.123 0.128 0.142
CP 0.940 0.937 0.952 0.938 0.946 0.945

In addition to the scenarios presented by Table 1, we investigated those with various link functions and random effects. For example, the results given in Table 2 were obtained with the same setups as those for Table 1 except that g(x) = x and μ0(t) = 2t. Such results all suggest that the proposed procedure perform well for practical situations. To further study how various link functions affected the estimation results, we also calculated the averaged sum of absolute residuals ( RES¯) for each scenario, defined as

Table 2.

Estimation results for θ with the link function g(x) = x.

θ0 n = 100 n = 200

0 0.2 0.5 0 0.2 0.5
(γ0, ξ0) = (0, 0)
Bias 0.008 0.005 −0.006 0.009 0.004 0.009
SEE 0.269 0.261 0.249 0.191 0.186 0.178
SSE 0.287 0.277 0.246 0.201 0.187 0.185
CP 0.932 0.928 0.948 0.939 0.953 0.940
(γ0, ξ0) = (0, 0.2)
Bias 0.035 0.041 0.047 0.042 0.036 0.040
SEE 0.259 0.254 0.245 0.186 0.181 0.174
SSE 0.282 0.265 0.257 0.191 0.184 0.184
CP 0.927 0.934 0.924 0.929 0.936 0.921
(γ0, ξ0) = (0.5, 0)
Bias −0.007 0.015 0.010 0.001 0.011 0.010
SEE 0.247 0.239 0.233 0.176 0.172 0.166
SSE 0.256 0.259 0.249 0.180 0.179 0.177
CP 0.939 0.930 0.927 0.939 0.933 0.936
(γ0, ξ0) = (0.5, 0.2)
Bias 0.052 0.051 0.045 0.040 0.051 0.042
SEE 0.244 0.239 0.231 0.174 0.171 0.166
SSE 0.252 0.258 0.237 0.178 0.171 0.169
CP 0.932 0.917 0.935 0.929 0.947 0.930
RES¯=1ni=1nj=1midM^i(Ti,j).

Table 3 presents the results obtained for scenarios represented by Tables 1 and 2 when n = 200, where the baseline mean function is common for Yi(t) given bi(t) and Zi. The results show that when the choice of g(·) is reasonable, such residuals are comparable whether the covariate effects are additive (for g(x) = log(x)) or multiplicative (for g(x) = x) to the response process.

Table 3.

Averaged sum of residuals based on results from Tables 1 and 2 when n = 200.

θ0 g(t) = log(t) g(t) = t

0 0.2 0.5 0 0.2 0.5
(γ0, ξ0)= (0,0) 3.134 3.584 4.260 3.116 3.473 4.147
(γ0, ξ0)= (0, 0.2) 3.311 3.795 4.525 3.288 3.678 4.435
(γ0, ξ0)= (0.5, 0) 4.115 4.872 5.982 4.117 4.658 5.736
(γ0, ξ0)= (0.5, 0.2) 4.404 5.177 6.367 4.379 5.056 6.255

One question of practical interest is that for longitudinal data when the observation process is informative, whether some existing procedure applies to the situations as considered by models (1)–(3). While there are limited procedures for regression analysis based on a class of transformation models for the response process, most of them model possible correlation between Yi(t) and Ñi(t) by incorporating a specific function of Ñi(s), st to the marginal mean of Yi(t) (Sun et al., 2005; Li et al., 2013; Zhao et al., 2013), for example, a function denoted by h(·) in Zhao et al. (2013). One possible drawback is that such applications are highly subject to the specific form of h(·), which cannot capture correlations of an arbitrary form. To illustrate this numerically, we considered both the proposed estimation procedure and the one given in Zhao et al. (2013). Note that the latter also considered a possible dependent terminal event time Di but assumed a noninformative Ci given Zi, For ease of comparison, we made Di > Ci in our scenarios and used each subject’s last observation time as Ci when applied the competing procedure. Table 4 presents the estimation results for θ obtained for g(x) = log(x), b1i=12(exp{0.5-vi}-exp{-0.5-vi}+Gi),b2i(t)=(t+1)exp{vi(t+1)}e0.5(t+1)-e-0.5(t+1), b3i = vi, dΛ0(t)=20tt+1{e0.5(t+1)-e-0.5(t+1)}, μ0(t) = exp{5t}, with vi and Gi being random numbers from the uniform distribution over (−0.5, 0.5) and the gamma distribution with mean 1 and variance 0.5, respectively. In the table, BIAS represents the estimated bias from the proposed estimate; BIAS1 and BIAS2 denote the estimated biases given by Zhao et al. (2013) using h(ℱit) = Ñ(t−) and h(ℱit) = 0, respectively. The results suggest that the proposed estimates still appear to be unbiased, but the competing method could give substantially biased estimates for θ when the correlations between Yi(t), Ni(t) and Ci introduced by bi(t) are misinterpreted by h(·) or totally ignored.

Table 4.

Estimation results of θ based on the proposed procedure and the one given by Zhao et al. (2013), when g(x) = log(x) and ξ0 = 0.

θ0 n = 100 n = 200

BIAS
BIAS1
BIAS2
BIAS
BIAS1
BIAS2
γ0 = 0.5
0 0.008 −0.140 −0.118 0.000 −0.148 −0.135
0.2 0.003 −0.162 −0.141 −0.002 −0.154 −0.143
0.5 −0.009 −0.167 −0.139 −0.011 −0.169 −0.149
γ0 = 0.8
0 −0.007 −0.208 −0.196 0.000 −0.212 −0.181
0.2 −0.005 −0.210 −0.202 −0.002 −0.224 −0.198
0.5 −0.009 −0.220 −0.192 −0.011 −0.246 −0.195

6. AN APPLICATION

In this section, we applied the proposed methodology described in the previous sections to longitudinal data arising from a skin cancer study conducted by the University of Wisconsin Comprehensive Cancer Center in Madison, Wisconsin (Li et al., 2011; Zhang et al., 2013). One main objective of this double-blind, placebo-controlled randomized Phase III clinical trial is to evaluate the effectiveness of 0.5g/m2/day PO difluoromethylornithine (DFMO) in reducing the recurrence rates of basal cell carcinoma (BCC) for patients with a history of skin cancers. At each visit, the numbers of BCC occurrences since the previous visit were recorded. Each patient was scheduled to be assessed every six months; however as expected, the actual observation times vary from patient to patient. Besides a patient’s treatment group (placebo or DFMO), the study also provided information on the number of prior skin cancer occurrences which is shown to be significantly related to the skin cancer recurrence process. For the analysis, we focus on the 290 patients with at least one observation. Among them, 161 patients had one or two skin cancer occurrences prior to the study, and the others had experienced more.

In the following, we consider covariates defined by Zi = (Zi1, Zi2)′, where Zi1 = 1 if patient i was given the DFMO treatment and Zi1 = 0 otherwise, and Zi2 = 1 if the patient had experienced more than two (up to 35) skin cancer occurrences and Zi2 = 0 if not, i = 1, …, 290. Yi(t) represents the total number of BCC occurrences observed up to time t. The longest follow-up time was scaled to be τ = 1, which corresponds to 1, 879 days in the original data set.

To apply the proposed estimation procedure, we assumed that the skin cancer recurrence process, the observation process and the hazard of censoring can be described by models (1)–(3), respectively. Following the notation above, the primary interest is to estimate θ1, the effect of DFMO. Table 5 presents the analysis results obtained by applying the proposed estimation procedure with W(t) = 1. We considered two link functions: g(x) = x and g(x) = log(x), and the results include the point estimates (Est.), the estimated standard errors (SEE), the estimated 90% confidence intervals (CI’s) and p-values for tests with the null hypotheses assuming no covariate effects. At the significance level of α = 0.1, the results suggest that DFMO has significantly reduced the recurrence rates of BCC, and a more severe skin cancer history appears to be positively correlated with the recurrence rate of skin cancer. Such results appear consistent with those concluded by Li et al. (2014) for both choices of link functions. In addition, the results also suggest that both the observation and follow-up times significantly depend on the covariates.

Table 5.

Analysis results for the skin cancer data.

Est. SEE 90% CI p-value

γ1 0.529 0.072 (0.410, 0.648) < 0.001
γ2 0.566 0.072 (0.448, 0.684) < 0.001
ξ1 1.203 0.171 (0.922, 1.484) < 0.001
ξ2 1.038 0.171 (0.757, 1.319) < 0.001
g(x) = x
θ1 −0.448 0.187 (−0.814, −0.082) 0.017
θ2 1.164 0.225 (0.723, 1.064) < 0.001
g(x) = log(x)
θ1 −0.225 0.123 (−0.427, −0.024) 0.066
θ2 0.972 0.118 (0.777, 1.167) < 0.001

To assess the adequacy of our models above, we applied the goodness-of-fit test derived in Section 4 and obtained the p-values of 0.801 and 0.383, respectively, for g(x) = x and g(x) = log(x). This suggests that while both of our link functions appear to be reasonable for the data, the former is preferred over the latter.

7. CONCLUDING REMARKS

This paper considers regression analysis of longitudinal data when both the observation and follow-up times may be informative about the underlying response process of interest. For the problem, we present a class of semiparametric transformation models for the response process which allow possible correlations to be characterized by time-dependent random effects. Comparing with existing models that assume either independence or structured dependence based on fixed forms or distributions, the proposed models provide flexibility for modeling both the underlying response process and its correlation to other processes. For parameter estimation, an easy-to-implement estimating equation approach is developed and both finite and asymptotic properties of the resulting estimators are established. In addition, the extensive simulation study indicated that the approach works well for practical situations and the approach is applied to a skin cancer study which motivated the research.

We note several possible directions for future work. First for simplicity, we assumed that the dependence between Yi(t) and Ni(t) in models (1)–(2) can be completely characterized by random effects bi(t) and covariates Zi(t). However in practice, one may want to incorporate more terms to the content of g(·) as well when additional information is available. For example, if it is known from pivotal trials or experiences that a longitudinal response depends on the length of period since subject i is last observed, one may consider modifying model (1) as follows:

E{Yi(t)Zi(t),bi(t)}=g{μ0(t)eθZi(t)+α(t-Ti,j)}b1i(t),

where j = max{k : Ti,kt} and Ti,j represents subject i’s last observation time. In such cases, the same methodology immediately applies for estimating θ and α together, by replacing θ and Zi(t) by (θ, α′)′ and (Zi(t)′, tTi,j)′, respectively, in the estimation procedure. Second, the focus of the article has been on regression analysis of the response process Yi(t), therefore, bi(t) was treated as a shared latent vector. However, if one is solely interested in calculating any correlation between Yi(t), Ni(t) and Ci at certain times, one may usually need a distribution assumption on bi(t) and apply some existing procedures for inference (Lipsitz et al., 2002; He et al., 2009; Sun et al., 2007, 2007b; Li et al., 2013). Other than the effects of bi(t), we have assumed the proportional rates and additive hazards models, respectively, on Ni(t) and Ci. In context of dependent processes, a procedure that is robust to such models is another interesting direction for future research.

Acknowledgments

The authors wish to thank the editor, the associate editor and the two reviewers for their constructive comments and suggestions that led to a great improvement of this manuscript. This work was partially supported by funds provided by National Science Foundation (grant DMS-1208978 to Sun), National Institutes of Health (grant 2 R37 AI054165 to Sun) and The University of North Carolina at Charlotte (to Sun and FRG 1-11172 to Li).

APPENDIX A

Proof of Theorem 1

To derive the asymptotic properties of the proposed estimator θ̂, we need the following regularity conditions.

  • (C1)

    {Ni(·),Yi(·),Ci,Zi(·)}i=1n are independent and identically distributed.

  • (C2)

    There exists a τ > 0 such that P(Ciτ ) > 0.

  • (C3)

    Both Ñi(t) and Yi(t) (0 ≤ tτ, i = 1, …, n) are bounded.

  • (C4)

    W(t) and Zi(·), i = 1, …, n, have bounded variations and W(t) converges almost surely to a deterministic function w(t) uniformly in t ∈ [0, τ].

  • (C5)

    Aθ=E0τW(t)g.{μ0(t)eθ0Zi(t)}{Zi(t)-ez(t)}2eθ0Zi(t)+η0Xi(t)μ0(t)dΛ1(t) and Ωη=E[0τ{Xi(t)-x¯(t)}2eη0Xi(t)dΛ1(t)] are both positive definite.

Define

U1(θ;η^)=i=1n0τW(t)Zi(t)×[Yi(t)dNi(t)-eη^Xi(t)g{μ^0(t)eθZi(t)}dΛ^1(t)]=0

and note that μ̂0(t) satisfies

i=1n[Yi(t)dNi(t)-eη^Xi(t)g{μ^0(t)eθZi(t)}dΛ^1(t)]=0,0tτ. (14)

Let

A^θ(θ)=-n-1U1(θ,η^)/θ,A^η(η)=-n-1U1(θ0,η)/η,Aθ=limnA^θ(θ0)andAη=limnA^η(η0).

The consistency of θ̂ and η̂ follows from the facts that U1(θ0; η̂) and Uη(η0) both tend to 0 in probability as n → ∞, and that Âθ (θ) and −n−1Uη(η)/η both converge uniformly to the positive definite matrices Aθ and Ωη over θ and η, respectively, in neighborhoods around the true values θ0 and η0. Then the Taylor series expansions of U1(θ̂; η̂) at (θ0; η̂) and (θ0, η0) yield n1/2(θ^-θ0)=Aθ-1n-1/2U1(θ0;η^)+op(1)=Aθ-1{n-1/2U1(θ0;η0)-Aηn1/2(η^-η0)}+op(1). The proof of Theorem 1 is sketched as follows:

  1. First, using some derivation operation to U1(θ; η̂) and (A.1), we can get
    A^θ(θ)=n-1i=1n0τW(t)g{μ^0(t)eθ^Zi(t)}×{Zi(t)-E^Z(t)}2eθZi(t)+η^Xi(t)dΛ^1(t).
  2. The use of Taylor expansions of U1(θ0; η0) and (A.1) at μ0(t) yield
    U1(θ0;η0)=i=1n0τw(t)(Zi(t)-ez(t))dMi(t)-i=1n0τw(t)(Zi(t)-ez(t))g{μ0(t)eθ0Zi(t)}×eη0Xi(t)d{Λ^1(t;η0)-Λ1(t)}+op(n1/2).
    It follows from (7) that
    Λ^1(t;η0)-Λ1(t)=1ni=1n0tdMi(t)s(0)(t)+op(n-1/2).
    Thus
    U1(θ0;η0)=i=1n(ξ1i-ξ2i)+op(n1/2), (15)

    where ξ1i=0τw(t)(Zi(t)-ez(t))dMi(t),ξ2i=0τw(t)d(t)s(0)(t)dMi(t) and d(t) = limn→∞ (t; θ0, η0).

  3. Differentiation of U1(θ0, η) and (A.1) with respect to η′ yields
    A^η(η)=n-1i=1n0τW(t)g{μ^0(t)eθ0Zi(t)}eηXi(t)×{Zi(t)-E^Z(t)}{Xi(t)-X¯(t;η)}dΛ^1(t;η) (A.1)
  4. According to equation (6) and the arguments similar to Lin et al. (2000), one can show that
    n1/2{η^-η0}=Ωη-1n-1/2i=1nζi+op(1) (A.2)

    where Ωη=E[0τ{Xi(t)-x¯(t)}2eη0Xi(t)dΛ1(t)] and ζi=0τ(Xi(t)-x¯(t))dMi(t).

Combining the results in steps (1)–(4), we have

U1(θ0;η^)=i=1n(ξ1i-ξ2i-ξ3i)+op(n1/2),

and hence

n(θ^-θ0)=Aθ-1n-1/2i=1n(ξ1i-ξ2i-ξ3i)+op(1), (A.3)

where ξ3i=0τAηΩη-1{Xi(t)-x¯(t)}dMi(t). Then it follows from the multivariate central limit theorem that the conclusions hold.

APPENDIX B

Proof of the null distribution of ℱ(t, z)

Define V(θ^,η^)=i=1n0tI(Zi(s)z)dM^i(s). By applying the Taylor expansion,

(t,x;θ^,η^)=n-1/2V(θ0,η0)+V(θ0,η0)nηn(η^-η0)+V(θ0,η^)nθn(θ^-θ0)+op(1).

By following arguments and manipulations similar to those in Appendix A, it can be shown

V(θ0,η0)=i=1n{u1i(t,z)-u2i(t,z)}+op(n1/2),

where u1i(t,z)=0t{I(Zi(s)z)-eI(s,z)}dMi(s),u2i(t,z)=0tΓ(s)s(0)(s)dMi(s) and Γ(t) = limn→∞ Γ̂(t; θ0, η0).

Also V(θ0,η0)nη and V(θ0,η^)nθ can be estimated by – η(t, z) and – θ(t, z), respectively. In addition, we obtained

n1/2{η^-η0}=Ωη-1n-1/2i=1nζi+op(1)

and

n(θ^-θ0)=Aθ-1n-1/2i=1n(ξ1i-ξ2i-ξ3i)+op(1),

from (A.2) and (A.3). Therefore, ℱ(t, z; θ̂, η̂) can be expressed as a sum of i.i.d. mean-zero terms for fixed t. By the multivariate central limit theorem, ℱ(t, z) converges in finite-dimensional distribution to a mean-zero Gaussian distribution. Since ℱ(t, z) is tight based on the empirical process theory, ℱ(t, z) converges weakly to a mean-zero Gaussian process that can be approximated by ℱ̂(t, z) given by equation (11).

Contributor Information

Yang Li, Department of Mathematics and Statistics, UNC Charlotte, Charlotte, NC 28223.

Yanqing Sun, Department of Mathematics and Statistics, UNC Charlotte, Charlotte, NC 28223.

References

  • 1.Diggle PJ, Liang KY, Zeger SL. The Analysis of Longitudinal Data. Oxford University Press; Oxford: 1994. [Google Scholar]
  • 2.Cheng SC, Wei LJ. Inferences for a semiparametric model with panel data. Biometrika. 2000;87:89–97. [Google Scholar]
  • 3.He X, Tong X, Sun J. Semiparametric analysis of panel count data with correlated observation and follow-up times. Lifetime Data Analysis. 2009;15:177–196. doi: 10.1007/s10985-008-9105-1. [DOI] [PubMed] [Google Scholar]
  • 4.Hu XJ, Sun J, Wei LJ. Regression parameter estimation from panel counts. Scandinavian Journal of Statistics. 2003;30:25–43. [Google Scholar]
  • 5.Huang CY, Wang MC, Zhang Y. Analysing panel count data with informative observation times. Biometrika. 2006;93:763–775. doi: 10.1093/biomet/93.4.763. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. Wiley; New York: 2002. [Google Scholar]
  • 7.Li N, Sun L, Sun J. Semiparametric transformation models for panel count data with dependent observation process. Statistics in Biosciences. 2010;2(2):191–210. doi: 10.1002/cjs.10118. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Li N, Zhao H, Sun J. Semiparametric transformation models for panel count data with correlated observation and follow-up times. Statistics in Medicine. 2013;32(17):3039–3054. doi: 10.1002/sim.5724. [DOI] [PubMed] [Google Scholar]
  • 9.Li Y, Zhao H, Sun J, Kim KM. Nonparametric tests for panel count data with unequal observation processes. Computational Statistics & Data Analysis. 2014;73:103–111. [Google Scholar]
  • 10.Lin DY, Oaks D, Ying Z. Additive hazards regression with current status data. Biometirka. 1998;85(2):289–298. [Google Scholar]
  • 11.Lin DY, Wei LJ, Yang I, Ying Z. Semiparametric regression for the mean and rate functions of recurrent events. Journal of the Royal Statistical Society, Series B. 2000;62:711–730. [Google Scholar]
  • 12.Lin DY, Ying Z. Semiparametric and nonparametric regression analysis of longitudinal data. Journal of American Statistical Association. 2001;96:103–126. [Google Scholar]
  • 13.Lipsitz SR, Fitzmaurice GM, Ibrahim JG, Gelber R, Lipshultz S. Parameter estimation in longitudinal studies with outcome-dependent follow-up. Biometrics. 2002;58:621–630. doi: 10.1111/j.0006-341x.2002.00621.x. [DOI] [PubMed] [Google Scholar]
  • 14.Sun J, Kalbfleisch JD. Estimation of the mean function of point processes based on panel count data. Statistica Sinica. 1995;5:279–289. [Google Scholar]
  • 15.Sun J, Park D-H, Sun L, Zhao X. Semiparametric regression analysis of longitudinal data with informative observation times. Journal of American Statistical Association. 2005;100:882–889. [Google Scholar]
  • 16.Sun J, Sun L, Liu D. Regression analysis of longitudinal data in the presence of informative observation and censoring times. Journal of the American Statistical Association. 2007;102:1397–1406. [Google Scholar]
  • 17.Sun J, Tong X, He X. Regression analysis of panel count data with dependent observation times. Biometrics. 2007b;63:1053–1059. doi: 10.1111/j.1541-0420.2007.00808.x. [DOI] [PubMed] [Google Scholar]
  • 18.Sun J, Wei LJ. Regression analysis of panel count data with covariate-dependent observation and censoring times. Journal of the Royal Statistical Society, Series B. 2000;62:293–302. [Google Scholar]
  • 19.Sun L, Song X, Zhou J, Liu L. Joint analysis of longitudinal data with informative observation times and a dependent terminal event. Journal of the American Statistical Association. 2013;107(498):688–700. [Google Scholar]
  • 20.Sun Y. Estimation of semiparametric regression model with longitudinal data. Lifetime Data Analysis. 2010;16(2):271–298. doi: 10.1007/s10985-009-9136-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Wellner JA, Zhang Y. Two estimators of the mean of a counting process with panel count data. Annals of Statistics. 2000;28:779–814. [Google Scholar]
  • 22.Wellner JA, Zhang Y. Two likelihood-based semiparametric estimation methods for panel count data with covariates. Annals of Statistics. 2007;35:2106–2142. [Google Scholar]
  • 23.Welsh AH, Lin X, Carroll RJ. Marginal longitudinal nonparametric regression: locality and efficiency of spline and kernel Methods. Journal of American Statistical Association. 2002;97:482–493. [Google Scholar]
  • 24.Zhang Y. A semiparametric pseudolikelihood estimation method for panel count data. Biometrika. 2002;89:39–48. [Google Scholar]
  • 25.Zhang Z, Sun J, Sun L. Statistical analysis of current status data with informative observation times. Statistics in Medicine. 2005;24:1399–1407. doi: 10.1002/sim.2001. [DOI] [PubMed] [Google Scholar]
  • 26.Zhao H, Li Y, Sun J. Analyzing Panel Count Data with Dependent Observation Process and a Terminal Event. The Canadian Journal of Statistics. 2013;41(1):174–191. [Google Scholar]
  • 27.Zhao X, Balakrishnan N, Sun J. Nonparametric inference based on panel count data. Test. 2011;20:1–42. [Google Scholar]
  • 28.Zhao X, Zhou J, Sun L. Semiparametric Transformation Models with Time-Varying Coefficients for Recurrent and Terminal Events. Biometrics. 2011;67:404–414. doi: 10.1111/j.1541-0420.2010.01458.x. [DOI] [PubMed] [Google Scholar]
  • 29.Zhao X, Tong X. Semiparametric regression analysis of panel count data with informative observation times. Computational Statistics and Data Analysis. 2011;55(1):291–300. [Google Scholar]
  • 30.Zhao X, Tong X, Sun J. Robust estimation for panel count data with informative observation times. Computational Statistics and Data Analysis. 2013b;57:33–40. [Google Scholar]

RESOURCES