Abstract
After initiation of treatment, HIV viral load has multiphasic changes, which indicates that the viral decay rate is a time-varying process. Mixed-effects models with different time-varying decay rate functions have been proposed in literature. However, there are two unresolved critical issues: (i) it is not clear which model is more appropriate for practical use, and (ii) the model random errors are commonly assumed to follow a normal distribution, which may be unrealistic and can obscure important features of within- and among-subject variations. Because asymmetry of HIV viral load data is still noticeable even after transformation, it is important to use a more general distribution family that enables the unrealistic normal assumption to be relaxed. We developed skew-elliptical (SE) Bayesian mixed-effects models by considering the model random errors to have an SE distribution. We compared the performance among five SE models that have different time-varying decay rate functions. For each model, we also contrasted the performance under different model random error assumption such as normal, Student-t, skew-normal or skew-t distribution. Two AIDS clinical trial data sets were used to illustrate the proposed models and methods. The results indicate that the model with a time-varying viral decay rate that has two exponential components is preferred. Among the four distribution assumptions, the skew-t and skew-normal models provided better fitting to the data than normal or Student-t model, suggesting that it is important to assume a model with a skewed distribution in order to achieve reasonable results when the data exhibit skewness.
Keywords: Bayesian analysis, HIV longitudinal dynamics, Mixed-effects models, Skew-elliptical distribution, Time-varying viral decay rate
1. Introduction
Mathematical modeling is an important tool for understanding the evolution of HIV viral load (number HIV-1 RNA copies in plasma) and interactions between HIV and its target cells. Due to the availability of highly active antiretroviral therapy (HAART) and sensitive methods of obtaining blood plasma HIV-1 RNA concentrations, it is possible to use viral load as a surrogate marker for the health status of HIV-infected individuals.
The viral load trajectory is complex and has multiple phases of change (Maldarelli et al., 2007; Perelson et al., 1997). Data from the AIDS clinical trial, ACTG5055 (Acosta et al., 2004) in Figure 1(a), show that: (i) within the first 2 weeks after the initial treatment, the viral load (transformed in natural log scale) dropped linearly and sharply, therefore, the change of viral load can be approximated by an exponential function; (ii) within the first 2–3 months but after the first 2 weeks, the relationship between the viral load and time was still linear but the slope became flatter, which indicates a slower decay rate; (iii) between the third to eighth month, the viral load either decreased more slowly, remained at a constant low level, or started to increase up to the level measured before treatment was initiated. The possible reasons for viral load rebound are development of resistance to the medications, and other clinical issues such as lack of adherence. There is no clear cutoff among the phases, not every subject will have all of these phases and the length of the phases may vary among individuals. Therefore, the associated decay rate in the models for the viral load trajectories is expected to vary over time and can be individually specific.
Figure 1.
Profile and histogram of viral load in natural log scale from a clinical trial study.
For the first phase of HIV viral load dynamics (i.e., the first 1 to 2 weeks), we can apply a uniexponential equation (Ho et al., 1995; Wei et al., 1995) as,
| (1) |
where V (t) is total viral load at time t, V (0) is the baseline viral load at t = 0 and λ is a constant viral change rate which is the speed of the loss of viral load after initiation of antiviral treatment. Although equation (1) can precisely describe the phenomenon of a linear decrease of logarithm transformed viral load within approximately one to two weeks since treatment, we cannot apply it to the whole trajectory because the viral load is only allowed to decrease at a constant rate in this equation. Besides that, there are at least three unsolved issues.
First, in order to use entire HIV follow up data, extended from equation (1), different models have been proposed in the literature, it is unclear which one is more appropriate.
Second, in mixed-effects model for longitudinal data analysis, random errors and/or random-effects are usually assumed to have a normal distribution. Although the normality assumption is satisfied in many situations, it may cause biased or misleading inference if the data include extreme values or show skewness with heavy tails, which are commonly seen in virological responses (Huang and Dagne, 2011; Sahu et al., 2003; Verbeke and Lesaffre, 1996). Figure 1(b) displays the histogram of repeated viral load in natural log scale for 44 subjects enrolled in the ACTG5055 trial. The skewness, which is still obvious even after the transformation, is positive and ranges from 0.8 and 2.15 at each of the follow up measurements. If the ratio between skewness value and standard error of skewness is greater than 2, the data may be regarded as having unignorable skewness (Gardner, 2001). In the ACTG5055 study, the ratio is 4, which indicates skewness needs to be accounted for.
Third, computation can be a challenge. Frequentist and Bayesian are two major approaches used in studies of HIV dynamics. In the frequentist approach, based on the maximum likelihood estimation (MLE), different extensions have been proposed, such as Laplace approximation of the numerical integrals (Beal and Sheiner, 1982; Lindstrom and Bates, 1990; Wu and Zhang, 2002), stochastic approximation EM (SAEM) algorithm (Kuhn and Lavielle, 2005; Lavielle et al., 2011), joint model via Monte Carlo EM algorithm (Liu and Wu, 2007; Wu, 2004) and asymptotic distribution of the maximum h-likelihood estimators (MHLE) (Commenges et al., 2011). The second approach is Bayesian mixed-effects modeling via Markov chain Monte Carlo (MCMC) (Huang et al., 2006; Huang and Dagne, 2011; Putter et al., 2002). The Bayesian approach is an efficient way to incorporate prior information, both point estimates and uncertainties (variances), into analysis to identify more unknown parameters in complex models.
Via Bayesian approach, the main focus of this paper is to provide a comprehensive comparison of five commonly used HIV dynamic models with skew-elliptical (SE) distribution in random error. The rest of the paper is organized as follows. Section 2 presents the HIV dynamic models that have a time-varying decay rate function, so they can be applied to the entire follow-up data. In Section 3, we describe a general Bayesian mixed-effects modeling approach. In Section 4, we present the motivated AIDS data and results of model comparisons. Section 5 includes the conclusion and discussion.
2. HIV dynamic models with time-varying decay rate function
As mentioned in Section 1, there is a multiphasic change in HIV viral load after the initiation of HAART. One potential interpretation of this phenomenon is that the process involves distinct populations with different homogenous behaviors. For example, the fast decreasing decay rate observed in the first phase is due to the treatment effect on productively infected CD4 cells, while the slower decay rate in the second phase is primarily due to the effect on the latently or long-lived infected cells (Perelson et al., 1997). However, some phenomena can’t be explained by this theory. For example, there can be large differences in mean decay rates in response to different treatment regimens: during the first week, the death of infected cells may be substantially slower during days 3–6 than during days 2–3 (Grossman et al., 1999). Following equation (1), Grossman et al.(1999) proposed an equation for viral load as:
| (2) |
where R is reproduction ratio and is a function of time, ρ is an adjustable parameter, τ is the average infection cycle time, υ(t) is the ratio between the viral load at time t and the baseline. Following Zhang and Wu (2011), equation (2) is equivalent to the following equation,
| (3) |
where is the average relative lost rate of the viral load V (t). λ(t) can be positive (if R < 1), zero (if R = 1), or negative (if R > 1). If λ(t) < 0, the decay rate λ(t) at time t actually is a growth rate. Therefore, by including a time-varying decay rate function λ(t), compared with equation (1), equation (3) is more flexible and can be applied to include the entire follow up data without the need to arbitrarily truncate the data.
A unified model with a time-varying viral decay rate function can be expressed as:
| (4) |
where y(t) is the natural logarithm transformation of the number of HIV-1 RNA copies per mL of plasma, ε is the measurement error, ln(V (0)) = β1 and is the macro-parameter for initial viral load in natural log scale.
Among the different decay rate functions proposed in literature, we select five representatives as follow (Dagne and Huang, 2012; Grossman et al., 1999; Wu, 2004; Zhang and Wu, 2011),
| (5) |
where the last one, λ(t) = υ[w(t), hi(t)], is a nonparametric function.
3. Bayesian mixed-effects models with skewed distribution
To account for the skewness observed in the data, the random errors in mixed-effects models can be assumed to follow an SE distribution (see Appendix A in detail). The SE distribution is a family of distributions that is not only mathematically tractable but also flexible in its possible shapes. Because in the SE family, skew-normal (SN), normal and Student-t distribution are all a special case of skew-t (ST) distribution, therefore, in this section, we present a general form of a mixed-effects model with an ST distribution under the Bayesian approach. A general mixed-effects model with an ST distribution can be expressed as,
| (6) |
yi = (yi1, …, yini)T with yij being the response value for the ith individual at the jth time (i = 1, …, n; j = 1, …, ni), gi(ti, βi) = (g(ti1, βi), …, g(tini, βi))T, ti = (ti1, …, tini)T, βi and β are individual-specific parameter vector and population parameter vector, respectively, g(·) and d(·) are linear or nonlinear known parametric functions, bi is normal random-effect vector with Σb being an unstructured covariance matrix. The vector of random errors εi = (εi1, …, εini)T follows a multivariate ST distribution with degrees of freedom ν, within-subject covariance matrix Σ and we usually can assume Σ = σ2Ini, and unknown ni × ni skewness diagonal matrix such that Δ = diag(δi1, …, δini), skewness parameter vector δi = (δi1, …, δini)T. In particular, if δi1 = ⋯ = δini ≙ δ, then Δ = δIni and δi = δ1ni, where 1ni = (1, …, 1)T, indicating that we are interested in skewness of overall data set.
Following discussion in Appendix A, to implement an MCMC procedure to model (6), by introducing one ni × 1 random vector wi, based on the stochastic representation, the model can be hierarchically formulated as follows.
| (7) |
where , tni,ν(μ, A) denotes the ni-variate Student-t distribution with parameters μ, A and degrees of freedom ν, I(w > 0) is an indicator function and w = |X0| with X0 ~ tni,ν(0, Ini). Note that the hierarchical models above under Bayesian framework will allow researchers to easily implement the methods using the freely available WinBUGS software (Lynn et al., 2000) and the computational effort for the model with an ST distribution is almost equivalent to that of the model with a Student-t distribution.
The unknown population parameters in the model (6) are θ = {β, σ2, Σb, ν, δ}, and we assume they are independent of one another. Under Bayesian framework, we also need to specify prior distributions for unknown parameters as follows.
| (8) |
where the mutually independent Normal (N), Inverse Gamma (IG), Exponential (Exp) and Inverse Wishart (IW) prior distributions are chosen to facilitate computations (Davidian and Giltinan, 1995). The super-parameter matrices ∧ and Ω can be assumed to be diagonal for convenient implementation.
Let π(.) be a prior density function, so π(θ) = π(β)π(σ2)π(Σb)π(ν)π(δ). Denote the observed data by 𝒟 = {yi, i = 1, …, n}, and f(·|·) as a conditional density function. Based on Bayesian inference, the posterior density of θ is proportional to the observed data and prior distribution as:
| (9) |
In general, the integral in (9) is of high dimension and does not have any closed form. Analytic approximations to the integral may not be sufficiently accurate. Therefore, it is prohibitive to directly calculate the posterior distribution of θ based on the observed data. As an alternative, MCMC procedures can be used to sample based on (9) by the Gibbs sampling along with the Metropolis-Hastings (M-H) algorithm.
4. Application: AIDS clinical trial data
4.1. AIDS clinical trial data and specific models
We used two AIDS clinical trials to explore the best fit among the models with different time-varying decay rate functions and different model error assumption such as normal, SN, Student-t and ST distributions. The first trial, ACTG5055 (Acosta et al., 2004), is the focus. Further, we used data from another clinical trial, ACTG398 (Pfister et al., 2003), to validate the conclusions obtained from ACTG5055.
ACTG5055 study was a phase I/II, randomized, open-label, 24-week comparative study. It included 44 HIV-1 infected patients who failed their first protease inhibitor (PI) treatment. Subjects were randomly assigned into one of the two arms. RNA viral load was measured (copies/mL) in blood samples collected at study days 0, 7, 14, 28, 56, 84, 112, 140 and 168 with a low limit of quantification of 50 copies/mL. HIV-1 RNA measures below this limit are not considered reliable. Following the simple substitution method of one half the detection limit for the detection (Helsel, 1990), we used 25 copies/mL if the viral load was 50 copies/mL or less.
ACTG398 study was a phase II trial that included 481 HIV-1 positive patients with prior exposure to approved PIs and loss of virological suppression. All patients were assigned to receive routine antiretroviral treatment (ART). Besides these medications, depending on the dose and type of PIs to which the patients previously exposed, they were selectively randomly assigned into one of four groups. HIV-1 RNA levels were measured at the time of entry into the study (day 0), at study weeks 2, 4, 8, 16, 24, 32, 40, and 48, every 8 weeks thereafter, and at the time of confirmed virological failure. The low limit of quantification is 100 copies/mL and the HIV-1 RNA measures below this limit are not considered reliable and 50 copies/mL was used instead. We draw two samples from ACTG398 based on the method of simple random sampling without replacement, one sample includes 44 subjects and the other includes 100 subjects. We also used all of the 481 subjects in ACTG398 in the model comparisons.
Figure 2 shows the measurements of viral load in natural log scale for four randomly selected patients from ACTG5055 and two sample data sets from ACTG398. We can see that viral load trajectories vary widely and they are substantially different across individuals. To account for this time-varying viral load change, we applied a mixed-effects model with a time-varying decay rate function, as discussed in Section 2. In addition, we assumed the model errors followed an ST distribution in order to make it flexible in dealing with the skewness observed in the data. The exact day of viral load measurement was used to compute study day in our analysis.
Figure 2.
Profiles of viral load in natural log scale for four randomly selected patients among ACTG5055 and ACTG398, respectively.
Under the general layout as model (4), corresponding to the five time-varying decay functions presented in Section 2, the mixed-effects models can be expressed as follow.
- Model I: Quadratic linear mixed-effects model:
where β = (β1, β2, β3)T and(10) - Model II: Nonlinear mixed-effects model (uniexponential plus a constant):
where β = (β1, β2, β3, β4)T and(11) - Model III: Nonlinear mixed-effects model (uniexponential plus a linear function):
where β = (β1, β2, β3, β4, β5)T and(12) - Model IV: Nonlinear mixed-effects model (two uniexponential):
where β = (β1, β2, β3, β4, β5)T and(13) - Model V: Semiparametric mixed-effects model:
where w(t) and hi(t) are unknown nonparametric smooth fixed-effects and random-effects functions, respectively, and hi(t) are iid realizations of a zero-mean stochastic process. Model V is a semiparametric mixed effects model if w(t) and hi(t) are modeled non-parametrically such as splines or local polynomials. There are several ways to approximate these nonparametric functions. Following the similar approach as Shi et al.(1996), Rice and Wu (2001) and Huang and Dagne (2010), we used natural cubic basis function instead of smoothing splines or kernel methods for two reasons: this method is more straightforward in application and we can select the bases by Akaike information criterion (AIC) or the Bayesian information criterion (BIC) to balance the goodness-of-fit and model complexity. A linear combination of base function can be expressed as:
where μp and ξiq (q ≤ p) are the unknown vectors of fixed and random coefficients, respectively. We set ψ0 = ϕ0 ≡ 1 and took the same natural cubic splines in the approximations with p ≤ q, based on the AIC and BIC values, selected as w(tij) + hi(tij) ≈ μ0 + μ1ψ1(tij) + μ2ψ2(tij) + ξi0, where p = 3 and q = 1. Model V, therefore, can be expressed as,(14)
where β = (β1, μ0, μ1, μ2)T and .(15)
In each of the five models above, besides the ST distribution assumption, the model random error can also be assumed to follow other more specific distributions as normal, SN and Student-t. We used several criteria to check the model fit by applying the models on the data mentioned above.
We first used deviance information criterion (DIC) (Spiegelhalter, 2002) to compare models. DIC is a generalization of AIC that can be directly obtained from WinBUGS (Lunn et al., 2000), and it consists of two components:
| (16) |
where D̄ = Eθ|Y [D(θ)] = Eθ|Y [−2 log p(Y |θ)] is the posterior mean of the deviance and measures “goodness of fit”, pD = D̄ −D(θ̄) is the effective number of parameters that indicates “complexity”. Therefore, DIC = “goodness of fit” + “complexity”. Same as AIC and BIC, the smaller the value of DIC, the better of the model fit. DIC is not intended for identification of the ‘correct’ model, but rather merely as a way to compare a collection of alternative formulations.
Because model comparisons are critical for our study, besides DIC, we also compared the values of expected predictive deviance (EPD) and residual sum of squares (RSS) that are obtained from each model. EPD is formulated by EPD = E{Σi, j(yrep, ij − yobs, ij)2}, where the predictive value yrep, ij is a replicate of the observed yobs, ij and the expectation is taken over the posterior distribution of the model parameters θ (Gelman et al., 2003). RSS is given by Σi, j(yobs, ij − yfitted, ij)2. The smaller the value of DIC, EPD and RSS, the better fit of the model to the data. Besides these statistical criteria, two types of plots, Quantile-Quantile (Q-Q) plot and observed values vs. fitted values, were also reported to give a visualized goodness-of-fit in the model comparisons.
To avoid very small (large) estimates, which may be unstable, we rescaled the original time t (days) so that the time scale was between 0 and 1. We used the entire follow up data in all of the models. In the Bayesian inferential approach, we also need to specify values of the hyper-parameters at the population level. Weakly informative prior distributions are taken for all the parameters: (i) for each component of fixed-effects vector of β, the prior was assumed to follow independent normal distribution as N(0, 100); (ii) for the scale parameter σ2, we assumed a limiting non-informative inverse gamma prior distribution as IG(0.01, 0.01), therefore, the mean is 1 and variance is 100; (iii) the prior for the variance-covariance matrix for the random-effect Σb was taken to be inverse Wishart distribution as IW(Ω, υ), the degree of freedom, υ = 5, and Ω is diagonal matrix with diagonal elements being 0.01; (iv) for the skewness parameter δ, we chose normal distribution; (v) the degree of freedom ν followed truncated exponential distribution with ν0 = 0.5
The MCMC sampler was implemented using WinBUGS package (Lunn et al., 2000), and the WinBUGS program code is available in Appendix B. The posterior means and quantiles were drawn after the collecting the final MCMC samples. We used one long chain. Convergence, which refers the algorithm has reached its equilibrium target distribution, was closely watched by using the standard tools within WinBUGS such as trace plots, the MC error and depicting the evolution of the ergodic means of a quantity over the number of iterations. After an initial 100,000 burn-in iterations, every 50th MCMC sample was retained from the next 200,000. Thus, we obtained 4,000 samples of targeted posterior distribution of the unknown parameters for statistical inference.
4.2. Results
We carry out model comparisons in two steps. Step 1: in Section 4.2.1, we determine when the model errors are assumed to have an ST distribution, among the five models presented in Section 4.1, which one has the best fit. Step 2: in Section 4.2.2, because normal, SN and Student-t distributions are all a special case of an ST distribution, focusing on the best model selected from Step 1, we compare the results based on random errors with the normal, SN, Student-t and ST distributions. The model comparisons are carried on A5055 data and confirmed by A398 data. Section 4.2.3 presents the results based on the best model selected.
4.2.1 Comparison of five models under an ST distribution
We should not directly compare the parameter values obtained from the models that have different components. However, because the models were applied to the same HIV viral load data, we could use DIC, EPD and RSS to find out which model had the best fit. Table 1 indicates that Model IV has the best fit. Among all of the data sets: ACTG5055, the two randomly selected samples from ACTG398 and ACTG398 that includes every subject, Model IV constantly has the lowest DIC, EPD and RSS. For example, in ACTG5055, the DIC value for Model IV is 21.7, while the DIC values are 1192.0, 401.1, 669.2, and 1015.7 in Models I, II, III and V, respectively.
Table 1.
DIC, EPD and RSS among the five models, random error is assumed to follow an ST distribution.
| Data set | Model I | Model II | Model III | Model IV | Model V | |
|---|---|---|---|---|---|---|
| ACTG5055: | DIC | 1192.0 | 401.1 | 669.2 | 21.7 | 1015.7 |
| All subjects | EPD | 0.33 | 0.12 | 0.15 | 0.05 | 0.20 |
| (n=44) | RSS | 59.2 | 20.7 | 27.1 | 8.4 | 35.3 |
| ACTG398: | DIC | 1239.4 | 1122.6 | 1162.9 | 576.9 | 1200.5 |
| Sample 1 | EPD | 3.17 | 3.48 | 3.63 | 1.04 | 3.04 |
| (n=44) | RSS | 396 | 436 | 454 | 134 | 381 |
| ACTG398: | DIC | 2264.9 | 1947.5 | 2855.3 | 1757.7 | 2204.5 |
| Sample 2 | EPD | 0.44 | 0.51 | 3.45 | 0.33 | 0.39 |
| (n=100) | RSS | 130.6 | 158.3 | 1083 | 99.39 | 116.7 |
| ACTG398: | DIC | 12900.6 | 10891.2 | 10763.4 | 8819.3 | 14217 |
| All subjects | EPD | 1.49 | 1.42 | 1.31 | 0.93 | 1.84 |
| (n=481) | RSS | 2665 | 2185 | 2059 | 1470 | 2703 |
4.2.2 Comparison of the best Model IV with four distributions of random error
For Model IV, we further investigated how different distributions of random error would affect the model fit and the DIC values are shown in Table 2 below. The model with either ST (ACTG5055, two samples of ACTG398) or SN (whole ACTG398) has the lowest DIC. Because SN has a simpler structure than ST, and the larger the degree of freedom, the closer the Student-t distribution is to the normal distribution, it is not surprising to see when the sample size is big (e.g. ACTG398, n=481), SN has a smaller DIC value than ST. We also evaluated EPD and RSS (not shown here) which provide the equivalent conclusion to that from DIC.
Table 2.
DIC values based on For Model IV with different distribution assumptions.
| Data set: | ||||
|---|---|---|---|---|
| Distribution | ACTG5055 (n=44) | ACTG398 (n=44) | ACTG398 (n=100) | ACTG398 (n=481) |
| Normal | 1133.3 | 961.4 | 2158.5 | 10222.0 |
| SN | 222.8 | 727.3 | 1937.2 | 7788.2 |
| Student-t | 1004.9 | 949.8 | 2144.3 | 8993.9 |
| ST | 21.7 | 576.9 | 1757.7 | 8819.3 |
We applied Model IV on ACTG5055 to further compare the estimation results obtained from different distributions. The posterior mean (PM), the corresponding standard deviation (SD) and 95% credible interval (CI) for fixed-effects parameters are presented in Table 3. We found that: (i) with the exception β5 based on the normal distribution, all of the other estimates were significant because the 95% CIs don’t include zero; (ii) for the variance σ2, the estimated value based on the SN (0.05) and ST (0.01) models were much smaller than the model with normal (1.15) or Student-t assumption (0.38); (iii) among all of the parameters estimated, the SD obtained from ST were the smallest; (iv) the estimates were similar between normal and Student-t model, but they were substantially different to those obtained from SN or ST model. For example, β2 based on normal and Student-t models was 34.67 and 38.10, respectively, while it was 24.53 and 27.89 in SN and ST models, respectively; (v) the skewness parameter δ was significantly positive in SN and ST models, confirming the positive skewness of the viral load in natural log scale as shown in Figure 1(a); (vi) compared to the model with normal or Student-t random error, the models with an SN or ST fit the data better. For example, in ACTG5055, for DIC values, 1133.3 (normal) vs. 222.8 (SN), 1004.9 (Student-t) vs. 21.7 (ST), it indicates that consideration of a departure from normality will improve the model fitting.
Table 3.
A summary of the estimated posterior mean (PM) of interested population (fixed-effects), precision and skewness parameters, the corresponding standard deviation (SD) and lower limit (LCI) and upper limit (UCI) of 95 % equal-trial credible interval (CI) from Model IV with normal, SN, Student-t and ST distributions of random error (based on ACTG5055 data).
| Model IV | β1 | β2 | β3 | β4 | β5 | σ2 | δ | DIC | EPD | RSS | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Normal | PM | 8.30 | 34.67 | 5.95 | 6.77 | 0.38 | 1.15 | – | 1133.3 | 2.29 | 1133.3 |
| LCI | 7.81 | 26.86 | 4.01 | 2.65 | −0.40 | 0.96 | – | ||||
| UCI | 8.77 | 43.04 | 8.54 | 12.92 | 1.01 | 1.37 | – | ||||
| SD | 0.26 | 4.15 | 1.26 | 2.52 | 0.35 | 0.11 | – | ||||
| SN | PM | 6.69 | 24.53 | 6.38 | 13.49 | 1.55 | 0.05 | 2.22 | 222.8 | 0.10 | 17.5 |
| LCI | 6.03 | 12.66 | 4.10 | 7.10 | 0.89 | 0.01 | 1.97 | ||||
| UCI | 7.40 | 35.03 | 11.22 | 20.19 | 2.15 | 0.16 | 2.51 | ||||
| SD | 0.35 | 5.69 | 1.72 | 3.85 | 0.37 | 0.04 | 0.14 | ||||
| Student-t | PM | 8.33 | 38.10 | 9.41 | 11.97 | 0.94 | 0.38 | – | 1004.9 | 2.19 | 420.8 |
| LCI | 7.84 | 30.31 | 7.34 | 8.99 | 0.66 | 0.28 | – | ||||
| UCI | 8.83 | 46.61 | 11.64 | 15.59 | 1.24 | 0.50 | – | ||||
| SD | 0.25 | 4.11 | 1.13 | 1.66 | 0.15 | 0.06 | – | ||||
| ST | PM | 7.18 | 27.89 | 7.19 | 13.69 | 1.73 | 0.01 | 1.17 | 21.7 | 0.05 | 8.4 |
| LCI | 6.66 | 21.53 | 6.34 | 11.72 | 1.55 | 0.00 | 0.99 | ||||
| UCI | 7.67 | 33.93 | 8.63 | 16.22 | 1.98 | 0.04 | 1.34 | ||||
| SD | 0.25 | 3.26 | 0.66 | 1.28 | 0.13 | 0.01 | 0.08 | ||||
Several diagnostic plots for goodness-of-fit are also applied. Firstly, we select three representative subjects corresponding to the three typical patterns trajectories displayed in Figure 1 (a). These three trajectory patterns represent (i) rapid decline followed by slow decrease, (ii) rapid decline followed by rebound and then decrease, and (iii) decline followed by rebound. The individual estimates of viral load trajectories are shown in Figure 3. The following findings are observed: (i) the estimated individual trajectories obtained from SN and ST fit the originally observed data much closer than those got from the model where the random error is assumed to be normal or Student-t ; (ii) the average SD obtained from ST is the smallest, which is 0.15, while the mean of SD for the individual estimation got from SN, N and Student-t is 0.22, 0.52 and 0.46, respectively. Note that the lack of smoothness in SN and ST model estimates of individual trajectories is understandable since a random component wi was incorporated in the expected function (see equation (7) for details) according to the stochastic representation feature of the SN and ST distributions for “chasing the data” to this extent.
Figure 3.
Individual estimates of viral load trajectories for three representative patients based on Model IV with normal, Student-t, SN and ST distributions. The observed values are indicated by circle.
We also applied two diagnostic plots: plot of the observed values versus the fitted values (Figure 4) and Q-Q plot (Figure 5). The findings agree with that from DIC criterion: the models with SN and ST distributions provided better fit to the observed data than the ones with normal or Student-t distribution. Based on the results from DIC, EPD, RSS and the diagnostic plots, we conclude that Model IV with an ST distribution fits the data better than the other combinations among the models with a different time-varying decay rate function and/or distribution assumption of random errors.
Figure 4.
The observed values versus fitted values of ln(RNA) based on Model IV with normal, Student-t, SN and ST distributions, respectively.
Figure 5.
Q-Q plots based on Model IV with normal, Student-t, SN and ST distributions, respectively.
4.2.3 Results based on Model IV with an ST distribution
Based on Model IV with an ST distribution, the estimated population decay rate function based on ACTG5055 data is
Because the estimated λ̂(t) is always positive, the population viral load would always decrease in this specific HIV/AIDS data set.
The individual time-varying decay rate function is given by,
where the individual estimated decay rate λ0302;(tij) is considered to be dependent on both subjects and time points. We found that the individual decay rate at initial treatment, λ̂(ti1), where ti1 = 0, was positively correlated with baseline viral load (Spearman correlation coefficient r = 0.769, p < 0.0001) and negatively associated with baseline CD4 cell count (r = −0.447, p = 0.0025). Overall, the individual decay rate, λ̂(tij), was positively associated with viral load (p < 0.0001) and negatively associated with CD4 cell count (p < 0.0001).
Because 30 ~ 60% (Havlir et al., 2000) of patients eventually will have viral rebound, it is important to have a model that can reasonably predict this type of treatment failure in the long term. Following Wu et al.(2008), we defined rebound as, comparing with the HIV-1 viral load (natural log transformed) from the previous measurement, if there was ≥ 1.15 increase at one time point or ≥ 0.46 increase at two or more consecutive time points. In ACTG5055, there were 11 (26.2%) subjects had rebound. There was no significant difference in the baseline viral load (natural log (RNA)) between the rebound and no rebound group (median was 9.18 and 8.78/mL, respectively, p = 0.8610), while the median of baseline CD4 cell count intended to be higher in the no rebound group than in the rebound group (285 vs. 253/mL, respectively, p = 0.1169).
The trend of the changes in decay rates during the treatment was different between the rebound and no rebound group (Figure 6). For example, every individual decay rate was positive in the no rebound group, while some individual decay rates in the rebound group became negative, especially after the 3rd month of the treatment, which corresponded to the viral load rebound.
Figure 6.
Profiles of viral load in natural log scale and decay rate in rebound and no rebound group.
Based on the results of Model IV under the ST distribution in ACTG5055 data, we also found that: (i) overall, the average value of individual decay rates, λ̂(tij), was bigger in the no rebound group (14.97) than in the rebound group (12.93); (ii) the initial individual decay rates, λ̂(ti1), were significantly bigger in the no rebound group than in the rebound group (mean is 53.16 and 40.95, respectively); (iii) λ̂(ti1) was significantly associated with the rebound status in the long term (OR = 0.703, 95% CI is 0.580 – 0.853, p = 0.0003) and this association was still significant even after controlling the baseline viral load and CD4 cell count (OR = 0.717, 95% CI is 0.588 – 0.875, p = 0.0010). (iv) the average individual decay rate at the last visit (λ̂(tini)) among the no rebound subjects was 4.67, while it was −2.28 in the rebound group which indicates the viral load actually increasing instead of decreasing in this group. Among these findings, the most interesting is the associated relationship between initial individual decay rate (λ̂(ti1)) and rebound status: the results indicated that the odds of rebound decreased about 30% with each 1 unit increased in the decay rate. This may be helpful for physicians to predict the long-term results based on the information at early stage of the disease.
5. Discussion
With an ST distribution assumption for model random errors to account skewness observed in viral load responses, we compared five commonly used mixed-effects models in HIV dynamics via the Bayesian approach. We also investigated the impact of the four distributions in the skew-elliptical family on the model fitting. The results indicate that with the ST distribution, there is potential gain of efficiency and accuracy in estimating certain parameters when the normality assumption does not apply to the data. The skew-elliptical modeling via the Bayesian approach proposed in this study can be easily carried out via the WinBUGS package. Because the proposed model is quite general in theory and accessible to the existing software, it will allow statisticians to apply this method in other fields.
After finding the best fitting model, we estimated the relationship between the individual viral decay rate and some clinical important variables. The initial individual decay rate was positively correlated with the baseline viral load and negatively associated with baseline CD4 cell count. We also found that, overall, the average individual decay rate was lower in the rebound group than in the no rebound group. A more interesting finding is the significant association between the initial individual decay rate and the rebound status in the long-term, even after controlling for the baseline viral load and CD4 cell count. This finding is clinically important because it may enable physicians to predict the long-term outcome based on the estimated decay rate at an early stage.
Using the model with a time-varying decay rate function has some advantages over the biphasical models. (i) In the biphasical models, the association between the first decay rate and baseline viral load could be positive (Notermans et al., 1998; Wu et al., 2004) or negative (Wu et al., 1999); no significant association was found between the rebound and the first decay rate either (Wu et al., 2008); (ii) although the second decay rate in the biphasical models is supposed to be associated with long-term treatment status such as rebound (Ding and Wu, 1999; Wu et al.,2005), no significant association was found between the second decay rate and the low-level viral replication in long term (Sedaghat et al., 2008; Wu et al., 2003).
This paper has some limitations. Usually, covariates are included in the mixed-effects model to control within- and between-subject variation, and CD4 cell count is a commonly used covariate in HIV dynamic models. However, in order to use the original proposed models in the comparisons, we did not include covariates such as CD4 cell count or demographic information. For the viral load, the values below the detectable level are usually considered as inaccurate. Instead of treating these values as censored, we computed them by half the value of the detectable level. Furthermore, the issue of missing values is not considered in this study either.
In conclusion, the skewness parameter in the model with SN or ST distribution assumption is significantly positive, which confirms the positive skewness observed in the viral load data even after natural log transformed. The model fit is the best in the model with skewed (SN or ST) distribution. Because estimated parameters can be considerably different between the models with skewed distribution and normal or Student-t distribution, it is important to account for skewness in the model when data exhibits noticeable skewness. Different models may yield different conclusions about the relationship between the individual decay rate with viral load, CD4 cell count and the rebound status in HIV dynamics, therefore, it is also critical to choose a reasonable model that can balance between complexity and utility.
Acknowledgements
The authors thank the Editor and anonymous referees for their insightful comments and helpful suggestions that led to an improvement of the article. This research was partially supported by NIAID/NIH grant AI080338 to Huang.
Appendix A. Skill-elliptical distributions
Skew-elliptical (SE) distribution is a parametric class of probability distributions that is extended from elliptical distribution by including an additional shape parameter for skewness. This class, which is usually obtained by using transformation and conditioning, contains many standard families such as multivariate skew-normal (SN) and skew-t (ST) distributions. Different versions of the multivariate SE distributions have been proposed. We adopt a class of multivariate SE distributions proposed by Sahu et al.(2003), which is obtained by using transformation and conditioning and contains multivariate ST, SN, Student-t and normal distributions as special cases. A k-dimensional random vector Y follows a k-variate SE distribution if its pdf is given by
| (A.1) |
where A = Σ + Δ2, μ is a location parameter vector, Σ is a covariance matrix, Δ is a skewness diagonal matrix with the skewness parameter vector δ = (δ1, δ2, …, δk)T ; V follows the elliptical distribution and the density generator function , with mν(u) being a function such that exists. The function mν(u) provides the kernel of the original elliptical density and may depend on the parameter ν. We denote this SE distribution by SE(μ, Σ, Δ; m(k)). Two examples of mν(u), leading to important special cases used throughout the paper, are mν(u) = exp(−u/2) and mν(u) = (u/ν)−(ν+k)/2, where ν > 0. These two expressions lead to the multivariate SN and ST distributions, respectively. In the latter case, ν corresponds to the degree of freedom parameter.
ST distribution
We briefly discuss a multivariate ST distribution introduced by Sahu et al.(2003) in this section. A k-dimensional random vector Y follows a k-variate ST distribution if its probability density function (pdf) is given by
| (A.2) |
We denote the k-variate t distribution with parameters μ, A and degrees of freedom ν by tk,ν(μ, A) and the corresponding pdf by tk,ν(y|μ, A) henceforth, V follows the t distribution tk,ν+k. We denote this distribution by STk,ν(μ, Σ, Δ). In particular, when Σ = σ2Ik and Δ = δIk, equation (A.2) simplifies to
| (A.3) |
where Tk,ν+k(·) denotes the cumulative distribution function (cdf) of tk,ν+k(0, Ik). However, unlike in the SN distribution below, the ST density can not be written as the product of univariate ST densities. Here Y are dependent but uncorrelated.
By the proposition of Sahu et al.(2003), the ST distribution of Y has a convenient stochastic representation as follows.
| (A.4) |
where X0 and X1 are two independent random vectors following tk,ν(0, Ik). Note that the expression (A.4) provides a convenience device for random number generation and for implementation purpose. Let w = |X0|, then w follows a k-dimensional standard t distribution tk,ν(0, Ik) truncated in the space w > 0 (i.e., the standard half-t distribution). Thus, a hierarchical representation of (A.4) is given by
| (A.5) |
where ω = (ν + wTw)/(ν + k).
The ST presented in equation (A.5) can be reduced to the following three special cases:
as ν → ∞ with probability of 1, it becomes an SN distribution with SNni (μ, Σ, Δ);
as Δ = 0, it becomes standard multivariate Student-t-distribution tni,ν(μ, Σ);
if both (i) and (ii) are satisfied, then it is a standard multivariate normal distribution Nni(μ, Σ);
SN distribution
We briefly discuss a multivariate SN distribution introduced by Sahu et al.(2003) in this section. A k-dimensional random vector Y follows a k-variate SN distribution, if its pdf is given by
| (A.6) |
where V ~ Nk{ΔA−1(y−μ), Ik−ΔA−1Δ}, and ϕk(·) is the pdf of Nk(0, Ik). We denote the above distribution by SNk(μ, Σ, Δ). An appealing feature of equation (A.6) is that it gives independent marginal when . The pdf (A.6) thus reduces to
| (A.7) |
where ϕ(·) and Φ(·) are the pdf and cdf of the standard normal distribution, respectively.
It is noted that when Δ = 0, the SN distribution reduces to usual normal distribution. In addition, the SN distribution is a special case of the ST distribution. That is, the ST distribution reduces to the SN distribution when the degree of freedom is large.
Appendix B. WinBUGS code for Model IV with ST distribution
## Begin of model
model
{
for (i in 1:N)
{
# Random effects of response model
b2[i,1]<-0
b2[i,2]<-0
b2[i,3]<-0
b2[i,4]<-0
b2[i,5]<-0
b[i,1:5]~dmnorm(b2[i,1:5],Omega2[,])
# Individual parameter estimates
nbeta1[i]<-beta[1]+b[i,1]
nbeta2[i]<-beta[2]+b[i,2]
nbeta3[i]<-beta[3]+b[i,3]
nbeta4[i]<-beta[4]+b[i,4]
nbeta5[i]<-beta[5]+b[i,5]
}
for (j in 1:M)
{
## LME model with ST distribution
beta1[j]<-beta[1]+b[y[j,4],1]
beta2[j]<-beta[2]+b[y[j,4],2]
beta3[j]<-beta[3]+b[y[j,4],3]
beta4[j]<-beta[4]+b[y[j,4],4] # y[j,4]= id
beta5[j]<-beta[5]+b[y[j,4],5]
decay[j]<-(beta2[j]*exp(-beta3[j]*y[j,6])
+ beta4[j]*exp(-beta5[j]*y[j,6])) # decay rate
dm1[j]<-beta1[j] -decay[j]*y[j,6] # y[,6]= time
w[j]~dt(0, 1,nu) I(0,)
mu[j]<-dm1[j]+delta*w[j] # ST distribution
aau[j]<-(nu+w[j]*w[j])/n1*eta
y[j,12]~dt(mu[j],aau[j],n1) # y[,12]=logeRNA
Y.pred[j]~dt(mu[j],aau[j],n1) # predicted values
# Fitted values and Residuals
fit[j] <- mu[j]
resid[j] <- y[j,12]-fit[j]
ssr.r[j] <- pow(resid[j],2) # squares of residuals
ssr.Y.obs[j] <- pow((Y.pred[j]-y[j,12]),2) # squares of predicted deviation
}
SSR<-sum(ssr.r[]) # SSR
SSR.pred<-mean(ssr.Y.obs[]) # EPD
# Prior distributions of the hyperparameters
#(1) Degree of freedom
nu0<-0.1
nu ~dexp(nu0) I(3,)
d1<-0.5*nu
n1<-nu+1
#(2) Coefficients
for (l in 1:5) {beta[l]~dnorm(0,0.01)}
#(3) Skewness parameter
delta ~ dnorm(0, 0.01)
#(4) Variance-covariance matrice
Omega2[1:5,1:5]~dwish(R2[,],5)
v2[1:5,1:5]<-inverse(Omega2[,])
#(5) Precision parameters
eta~dgamma(0.01,0.01)
sigma <-1/eta
}
## End of model
References
- Acosta EP, Wu H, Walawander A, Eron J, Pettinelli C, Yu S, Gerber JG. Comparison of two indinavir/ritonavir regimens in treatment-experienced HIV-infected individuals. Journal of Acquired Immune Deficiency Syndromes. 2004;37:1358–1366. doi: 10.1097/00126334-200411010-00004. [DOI] [PubMed] [Google Scholar]
- Beal SL, Sheiner LB. Estimating population kinetics. Critical Review in Biomedical Engineering. 1982;8:192–222. [PubMed] [Google Scholar]
- Commenges D, Jolly D, Drylewicz J, Putter H, Thiebaut R. Inference in HIV dynamics models via hierarchical likelihood. Biometrics. 2011;55(1):446–456. [Google Scholar]
- Dagne G, Huang Y. Mixed-effects Tobit joint models for longitudinal data with skewness, detection limits and measurement errors. Journal of Probability and Statistics. 2012;2012(2012) Article ID 614102. [Google Scholar]
- Davidian M, Giltinan DM. Nonlinear Models for Repeated Measurement Data. London: Chapman & Hall; 1995. [Google Scholar]
- Ding AA, Wu H. Relationships between antiviral treatment effects and biphasic viral decay rates in modeling HIV dynamics. Mathematical Biosciences. 1999;160:63–82. doi: 10.1016/s0025-5564(99)00021-8. [DOI] [PubMed] [Google Scholar]
- Gelman A, Carline JB, Stern HS, Rubin DB. Bayesian data analysis. London: Chapman & Hall/CRC; 2003. [Google Scholar]
- Gardner RC. Pyschological statistics using SPSS for Windows. McGraw-Hill; 2001. [Google Scholar]
- Grossman Z, Polis M, Feinberg M, Grossman Z, Levi I, Jankelevich S, Yarchoan R, Boon J, de Wolf F, Lange JMA, Goudsmit J, Dimitrov DS, Paul WE. Ongoing HIV dissemination during HAART. Nature Medicine. 1999;5(10):1099–1104. doi: 10.1038/13410. [DOI] [PubMed] [Google Scholar]
- Havlir DV, Hellmann NS, Petropoulos CJ, Whitcomb JM, Collier AC, Hirsch MS, Richman DD. Drug susceptibility in HIV infection after viral rebound in patients receiving Indinavir-containing regimens. Journal of the American Medical Association. 2000;283:229–234. doi: 10.1001/jama.283.2.229. [DOI] [PubMed] [Google Scholar]
- Helsel DR. Less than obvious-statistical treatment of data below the detection limit. Environmental Science and Technology. 1990;24(12):1766–1774. [Google Scholar]
- Ho DD, Neumann AU, Perelson AS, Chen W, Leonard JM, Markowitz M. Rapid turnover of plasma virions and CD4 lymphocytes in HIV-1 infection. Nature. 1995;373:123–126. doi: 10.1038/373123a0. [DOI] [PubMed] [Google Scholar]
- Huang Y, Liu D, Wu H. Hierarchical Bayesian Methods for Estimation of Parameters in a Longitudinal HIV Dynamic System. Biometrics. 2006;62(2):413–423. doi: 10.1111/j.1541-0420.2005.00447.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Huang Y, Dagne G. Skew-normal Bayesian nonlinear mixed-effects models with application to AIDS studies. Statistics in Medicine. 2010;29:2384–2398. doi: 10.1002/sim.3996. [DOI] [PubMed] [Google Scholar]
- Huang Y, Dagne G. A Bayesian approach to joint mixed-effects models with a skew-normal distribution and measurement errors in covariates. Biometrics. 2011;67:260–269. doi: 10.1111/j.1541-0420.2010.01425.x. [DOI] [PubMed] [Google Scholar]
- Kuhn E, Lavielle M. Maximum likelihood estimation n nonlinear mixed effects models. Computational Statistics and Data Analysis. 2005;49:1020–1038. [Google Scholar]
- Lavielle M, Samson A, Karine-Fermin A, Mentre E. Maximum likelihood estimation of long-term HIV dynamic models and antiviral response. Biometrics. 2011;67(1):250–259. doi: 10.1111/j.1541-0420.2010.01422.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Lindstrom M, Bates D. Nonlinear mixed effects models for repeated measures data. Biometrics. 1990;46:673–687. [PubMed] [Google Scholar]
- Liu W, Wu L. Simultaneous inference for semiparametric nonlinear mixed-effects models with covariate measurement errors and missing responses. Biometric. 2007;63:342–350. doi: 10.1111/j.1541-0420.2006.00687.x. [DOI] [PubMed] [Google Scholar]
- Lunn DJ, Thomas A, Best N, Spiegelhalter D. WinBUGS – a Bayesian modelling framework: concepts, structure, an extensibility. Statistics and Computing. 2000;10:325–337. [Google Scholar]
- Maldarelli F, Palmer S, King MS, Wiegand A, Polis MA, Mican J, Kovacs JA, Mellors JM. ART suppresses plasma HIV-1 RNA to a stable set point predicted by pretherapy viremia. PloS Clinical Trials. 2007;3(4):e46–e87. doi: 10.1371/journal.ppat.0030046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Notermans DW, Goudsmit J, Danner SA, De-Wolf F, Perelson AS, Mittler J. Rate of HIV-1 decline following antiretroviral therapy is related to viral load at baseline and drug regimen. AIDS. 1998;12:1483–1490. doi: 10.1097/00002030-199812000-00010. [DOI] [PubMed] [Google Scholar]
- Perelson AS, Essunger P, Cao Y, Vesane M, Hurley A, Saksela K, Markowltz M, Ho DD. Decay characteristics of HIV-1-infected compartments during combination therapy. Nature. 1997;387:188–191. doi: 10.1038/387188a0. [DOI] [PubMed] [Google Scholar]
- Pfister M, Labbe L, Hammer SM, Mellors H, Bennett KK, Rosenkranz S, Sheiner LB the AIDS clinical trial group protocol 398 investigators. Population pharmacokinetics and pharmacodynamics of Efavirenz, Nelfinavir and Indinavir: adult AIDS clinical trial group study 398. Antimicrobial Agents and Chemotherapy. 47(1):130–137. doi: 10.1128/AAC.47.1.130-137.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Putter H, Heisterkamp SH, Lange JMA, Wolf F. A Bayesian approach to parameter estimation in HIV dynamic models. Statistics in Medicine. 2002;21:2199–2214. doi: 10.1002/sim.1211. [DOI] [PubMed] [Google Scholar]
- Rice JA, Wu CO. Nonparametric mixed-effects models for unequally sampled noisy curves. Biometrics. 2001;57:253–259. doi: 10.1111/j.0006-341x.2001.00253.x. [DOI] [PubMed] [Google Scholar]
- Sahu SK, Dey DK, Branco MD. A new class of multivariate skew distributions with applications to Bayesian regression models. The Canadian Journal of Statistics. 2003;31:129–150. [Google Scholar]
- Shi M, Weiss RE, Taylor JMG. An analysis of pediatric CD4+ counts for acquired immune deficiency syndrome using flexible random curves. Applied Statistics. 1996;45:151163. [Google Scholar]
- Spiegelhalter DJ, Best NG, Carlin BP, Van der Linde A. Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society, Series B. 2002;64:583–639. (with Discussion). [Google Scholar]
- Verbeke G, Lesaffre E. A linear mixed-effects model with heterogeneity in random-effects population. Journal of the American Statistical Association. 1996;91:217–221. [Google Scholar]
- Wei X, Ghosh SK, Taylor ME, Johnson VA, Emini EA, Deutsch PD, et al. Viral dynamics in human immunodeficiency virus type 1 infection. Nature. 1995;373(12):117–122. doi: 10.1038/373117a0. [DOI] [PubMed] [Google Scholar]
- Wu H, Kuritzkes DR, McClernon DR, Kessler H, Connick E, Landay A, Lederman MM. Characterization of viral dynamics in human immunodeficiency virus type 1-patients treated with combination antiretroviral therapy: relationship to host factors, cellular restoration, and virologic end points. The Journal of Infectious Diseases. 1999;179:799–807. doi: 10.1086/314670. [DOI] [PubMed] [Google Scholar]
- Wu H, Zhang J-T. The study of long-term HIV dynamics using semi-parametric non-linear mixed-effects models. Statistics in Medicine. 2002;21:3655–3675. doi: 10.1002/sim.1317. [DOI] [PubMed] [Google Scholar]
- Wu H, Zhao C, Liang H. Comparison of linear, nonlinear and semiparametric mixed-effects models for estimating HIV dynamics parameters. Biometrical Journal. 2004;6:233–245. [Google Scholar]
- Wu L. Simultaneous inference for longitudinal data with detection limits and covariates measured with errors, with application to AIDS studies. Statistics in Medicine. 2004;23:1715–1731. doi: 10.1002/sim.1748. [DOI] [PubMed] [Google Scholar]
- Wu H, Huang Y, Acosta EP, Rosenkranzm SL, Kuritzkesm DR, Eronm JJ, Perelson AS, Gerber JG. Modeling long-term HIV dynamics and antiretroviral response. Journal of AIDS. 2005;39:272–283. doi: 10.1097/01.qai.0000165907.04710.da. [DOI] [PubMed] [Google Scholar]
- Wu L, Hu XJ, Wu H. Joint inference for nonlinear mixed-effects models and time to event at the presence of missing data. Biostatistics. 2008;9(2):308–320. doi: 10.1093/biostatistics/kxm029. [DOI] [PubMed] [Google Scholar]
- Zhang J, WU H. Modeling HIV dynamics using unified mixed-effects models. American Journal of Mathematical and Management Sciences. 2011;30(2):83–111. [Google Scholar]






