Evaluation of treatment efficacy using a Bayesian mixture piecewise linear model of longitudinal biomarkers

Abstract

Prostate-specific antigen (PSA) is a widely used marker in clinical trials for patients with prostate cancer. We develop a mixture model to estimate longitudinal PSA trajectory in response to treatment. The model accommodates subjects responding and not responding to therapy through a mixture of two functions. A responder is described by a piecewise linear function, represented by an intercept, a PSA decline rate, a period of PSA decline and a PSA rising rate; a non-responder is described by an increasing linear function with an intercept and a PSA rising rate. Each trajectory is classified as a linear or a piecewise linear function with a certain probability, and the weighted average of these two functions sufficiently characterizes a variety of patterns of P SA trajectories. Furthermore, this mixture structure enables us to derive clinically useful endpoints such as a response rate and time-to-progression, as well as biologically meaningful endpoints such as a cancer cell killing fraction and tumor growth delay. We compare our model to the most commonly used dynamic model in the literature and show its advantages. Finally, we illustrate our approach using data from two multi-center prostate cancer trials. The R code used to produce the analyses reported in this paper is available on request.

1. Introduction

Prostate-specific antigen (PSA), a serine protease normally produced in the prostate, is a widely used marker in clinical trials for patients with prostate cancer. Typically it becomes elevated when the prostatic epithelium undergoes malignant transformation. The longitudinal trajectory for PSA levels is often non-linear in clinical trials. PSA levels are conventionally categorized as complete response, partial response, no change, or progressive disease [1]. For instance, a partial response is determined if the PSA level is decreased by at least 50% from the baseline level, and remained under 50% of the baseline level for at least 28 days. However, using these categorized endpoints often leads to biased estimates [2] and underpowered comparisons [3–5].

In this paper, we propose using continuous longitudinal PSA data to evaluate treatment effects. Motivated by a Phase III multi-center prostate cancer trial [1], we observe that, in many cases, there is a decline in PSA immediately after therapy followed by a rise in the recurrence of the cancer, and the decline in PSA represents that a subject responded to therapy. There has been extensive literature on fitting such non-linear PSA trajectories using a piecewise linear model (PWL) with a random changepoint [6–9]. Similar PWL models have been applied to other studies related to HIV [10], cognitive function [11], AIDS [12] and xenograft experiments [13]. However, we also observe that not all patients have a decline in PSA following therapy. For these patients, the longitudinal PSA data increase until disease progression as a sign of not responding to therapy. Therefore, it is important for a model to accommodate both of these trajectories for responders and non-responders. The most commonly seen mixture model in the literature consists of a mixture of a 1-parameter intercept-only function and a 3-parameter PWL function [10, 14–16]. These two functions are probably adequate to model longitudinal trajectories in their contexts. For example, a flat trajectory is expected for controls in the ovarian cancer screening study [14]. However, they will not be appropriate for modelling longitudinal efficacy biomarkers following treatment.

To accommodate responders and non-responders, we consider a model with an additional slope parameter in both the PWL and linear functions. This additional parameter allows an initial decline in PSA among responders or a linear increase among non-responders. For a responder, the PSA trajectory is described by a piecewise linear function, represented by an intercept, a PSA decline rate, a period of PSA decline and a PSA rising rate. For a non-responder, the trajectory is described by an increasing linear function with an intercept and a PSA rising rate. Using Bayesian inference, a subject’s trajectory is classified into a linear or a PWL function with a certain probability. This probability estimates how likely it is that the subject responds to therapy. Furthermore, this additional slope parameter offers a greater flexibility for modelling a variety of patterns of non-linear PSA trajectories as indicated in Figure 1. This figure depicts hypothetical PSA trajectories simulated from our mixture model with a mixture probability ranging from 0 to 1, at 0.2 intervals. When the mixture probability is 0, the trajectory is an increasing linear function of time; when the mixture probability is 1, the trajectory is a PWL function with a decline phase. Different mixture probabilities between 0 and 1 result in different patterns of PSA trajectories.

Figure 1.

Hypothetical non-linear PSA trajectories following treatment. Each curve is an average over 1000 curves simulated from a mixture model with a mixture probability, p, ranging from 0 to 1, at 0.2 intervals, a fixed intercept at 7, and fixed normal distributions for the random changepoint, slope before and after the change point.

In addition to providing a sufficient fit to non-linear trajectories, this mixture structure also enables us to derive clinically useful endpoints such as a response rate and time-to-progression, as well as biologically meaningful endpoints, such as a cancer cell killing fraction and tumor growth delay.

The remainder of this article is organized as follows. In Section 2, we develop a Bayesian mixture hierarchical longitudinal model, derive clinically and biologically meaningful endpoints and address the non-constant variance issue. In Section 3 we present simulation studies and compare our mixture model to a commonly used dynamic model. Finally, in Section 4 we analyze the prostate cancer trial. We conclude the article with a brief discussion.

2. Mixture Model

Let y_ij be the log₂ observed PSA data of subject i (i = 1, · · · , n) at time t_ij(j = 1, · · · , n_i). Log transformation is commonly used to obtain linear profiles before and after the nadir (the lowest PSA measurement), and the base 2 logarithms are related to PSA doubling times [8]. The observed PSA data are measured with error, and can be described by y_ij = μ_zi,ij + ε_ij , where ε_ij ~ N(0, σ²). An unobserved indicator z_i is included in the mean function to distinguish responders and non-responders. If z_i = 1, subject i is a responder, and the mean function of the log² PSA profile is described by a piecewise linear (PWL) function, which is given by

$${\mu }_{1,ij}=a_i+b_{1i}{t}_{ij}\mathrm{I}(t_{ij}\le {\tau }_i)+b_{1i}{\tau }_i\mathrm{I}(t_{ij}>{\tau }_i)+b_{2i}{(t_{ij}-{\tau }_i)}_{+}$$ (1)

where functions x₊ is defined as max(x, 0). Here, a_i is the log₂ measurement at baseline, τ_i is the unobserved changepoint (period of PSA decline), b_1i is the rate of decline in PSA, and b_2i is the PSA rising rate. If z_i = 0, subject i is a non-responder, and the mean function is reduced to a linear function

$${\mu }_{0,ij}=a_i+b_{2i}{t}_{ij}$$ (2)

Subject i has four parameters (a_i, b_1i, τ_i, b_2i) with z_i = 1, and only has two parameters (a_i, b_2i) with z_i = 0.

2.1. Hierarchical Structure

Both mean functions in (1) and (2) include the subject-specific intercept, a_i, which represents the baseline data (log₂ transformed) common to all treatment groups. Hence we will borrow strength across all subjects (in all treatment groups) for estimation of this parameter. Specifically, for i = 1, · · · , n, we consider

$$a_i\mid \alpha ,{\sigma }_a^2~N(\alpha ,{\sigma }_a^2)$$

where α and ${\sigma }_a^2$ are the population-level parameters. Borrowing the most information across all subjects and all treatment groups will gain precision in parameter estimations.

The log₂ PSA decline rate b_1i and period of decline τ_i are specific to z_i = 1, and it is very likely that they are affected by treatments. Therefore, we only allow pooling information across subjects with z_i = 1 in the same treatment group for the estimation of these parameters. Let g_i be the treatment applied to subject i out of k = 1, . . . , K treatments (hence g_i ∈ {1, . . . , K}), we consider

$$b_{1i}\mid {\beta }_{1g_i},{\sigma }_{b_{1g_i}}^2,z_i=1~N({\beta }_{1k},{\sigma }_{b_{1k}}^2)\mathrm{I}(b_{1i}<0)$$ (3)

$${\tau }_i\mid {\mu }_{\tau g_i},{\sigma }_{\tau g_i}^2,z_i=1~N({\mu }_{\tau k},{\sigma }_{\tau k}^2)\mathrm{I}({\tau }_i>0)$$ (4)

where β_1k, μ_τk, ${\sigma }_{b_{1k}}^2$, and ${\sigma }_{\tau k}^2$ are treatment-level parameters. The restrictions of b_1i < 0 and τ_i > 0 suggest that the log₂ PSA profile for a responder is characterized by an initial PSA decline followed by a rebound of PSA.

The parameter b_2i also appears in both z_i = 1 and z_i = 0. It represents a log₂ PSA growth rate when z_i = 0 and a regrowth rate when z_i = 1. To allow comparisons between treatment arms, we consider

$$b_{2i}\mid {\beta }_{2g_i},{\sigma }_{b_{2g_i}}^2~N({\beta }_{2k},{\sigma }_{b_{2k}}^2)$$ (5)

Additionally, indicators z_i (i = 1, · · · , n) are assumed to follow a Bernoulli distribution given by

$$z_i~\text{Bern}(n_{g_i},P_{g_i})$$ (6)

where n_gi is the number of subjects and P_gi is the response rate in treatment g_i (g_i ∈ {1, . . . , K}).

The hierarchical structures from (3)-(6) allow us to directly make comparisons between treatment groups using treatment-level parameters such as P_k, β_1k, μ_τk β_2k.

2.2. Posterior Computation

The Markov chain Monte Carlo procedure for estimating the posterior distributions is implemented by repeatedly drawing samples from the full conditional distributions of the parameters. The full conditional distributions for all parameters except τ_i and z_i are straightforward to compute due to the conjugacy, assuming posterior distributions are normal for the mean parameters and gamma for the precision parameters. The slice sampling algorithm developed by Neal [17] is used in sampling the changepoint τ_i. Updating the class indicator z_i requires an algorithm that can move between different parameter spaces, that is, between a space with two parameters (a_i, b_2i) and a space with four parameters (a_i, b_1i, τ_i, b_2i). We apply the pseudo-prior method of Carlin et al. [18] and the reversible jump procedure of Green [19] that allow such moving. Details of the calculation are given in Appendix A. With the prior restrictions in (3) and (4), these algorithms can move between z_i = 0 and z_i = 1 efficiently. When no changepoint is present in the trajectory, τ_i is not identifiable [20], and in this situation the algorithm will randomly pick a positive τ_i. With the constraint of b_1i < 0 and τ_i > 0 , the algorithm is able to correctly identify the pattern of z_i = 0 due to the absence of a negative pre-nadir slope.

2.3. Endpoints for Drug Efficacy

An important feature of our mixture model is its capability to derive clinically useful endpoints such as the response rate and time to PSA progression (TTP).

As defined in most clinical protocols with a PSA endpoint, a subject is classified as a responder if the subject experiences a decline in PSA following treatment, although the reduction threshold varies across protocols. In our mixture model, the log₂ PSA profile of subject i is classified as z_i = 0 or z_i = 1 at each MCMC iteration, and the proportion of times the profile is classified as z_i = 1 (corresponds to a PWL function) across all MCMC iterations estimates the probability that subject is a responder. Clinically, this probability may help guide physicians in developing individualized treatments. For a treatment k, its response rate, P_k can be calculated using z_i (∀i; g_i = k) by the beta-binomial conjugacy as shown in (6). This P_k is the preferred endpoint of response rate in Phase II trials.

Additionally, we derive the TTP endpoint as a X% increase in PSA taking the nadir (smallest value) as reference. We define TTP_k (k = 1, · · · ,K,) as

$${\text{TTP}}_k=P_k{\mu }_{\tau k}+{log}_2(1+X\%)/{\beta }_{2k}$$ (7)

At the subject level, TTP is the sum of two time periods: the time from the start of treatment to nadir and the time from nadir to progression. The first time period is the period of PSA decline, τ_i, if z_i = 1 and zero otherwise; the second time period, by simple algebra, equals to log₂(1 + X%)/b_2i. At the treatment level, TTP_k is defined in a similar fashion except that subject-level parameters are replaced by treatment-level parameters as shown in (7), in which the averaged treatment responding time is estimated as μ_τk weighted by P_k, the proportion of responders in treatment k.

If PSA is a surrogate for tumor size such that a decline (increase) of PSA values indicates regression (growth) of the tumor. Also, if treated tumors can be assumed to regrow at the same rate as untreated tumors (i.e., β₂₁ = · · · = β_2K = β₂) [21, 22], we can define cancer cell killing fraction (KF), a cell surviving fraction (LSF) and a tumor growth delay (TGD) respectively as

$$\begin{gathered} {\text{LSF}}_k=-P_k\times {\mu }_{\tau k}({\beta }_2-{\beta }_{1k}), \\ {\text{KF}}_k=1-2^{{\text{LSF}}_k}, \\ {\text{TGD}}_k=-{\text{LSF}}_k/{\beta }_2 \end{gathered}$$

These clinical and biological endpoints are all defined as a function of our model parameters. Using MCMC draws of model parameters, these endpoints can be easily calculated and compared between treatment groups.

2.4. Non-constant Error Variance

Since laboratory assays are known to have reduced precision at lower levels, we are concerned with the assumption that var(y_ij) = σ². To check this assumption, we modelled the log₂ variance as a linear function of the observed y_ij [8, 23]. Additionally, to improve model performance, we normalized longitudinal data of subject i by its baseline value, y_i1, specifically, ${\epsilon }_{ij}~N(0,{\sigma }_{ij}^2)$, where

$${log}_2({\sigma }_{ij}^2)={\sigma }^2-2r(y_{ij}-y_{i1})$$

If r = 0, data have a variance σ², which is independent of the marker value; if r > 0, the measurement variance decreases with the increasing marker level, and vice versa. Further details on MCMC sampling are given in Appendix B.

3. Simulation Studies

3.1. Performance of our mixture model

We carried out simulations to investigate the performance of our proposed mixture model defined in Section 2 (denoted as Mixture model). We simulated two groups, each containing 40 subjects, with each subject having measurements at baseline, at 2 weeks, monthly for 6 months, at 3-month intervals until the second year. The true parameter values were chosen to mimic the real data example, given by

• a_i ~ N(7, 2²) and σ² = 0.4².

• Arm I: P₁ = 0.6, b_1i ~ N(−0.8, 0.16²), τ_i ~ N(3, 1.5²), b_2i ~ N(0.4, 0.1²). Arm II: P₂ = 0.8, b_1i ~ N(−1.2, 0.16²), τ_i ~ N(5, 1.5²), b_2i ~ N(0.2, 0.1²). In both arms, b_1i is truncated below zero and τ_i and b_2i are truncated above zero.

In the simulations, data were truncated at a progression point, determined on the basis of a moving average of three consecutive values of PSA. Progression was defined as an increase in PSA equal to, or larger than, 50% above the lowest prior moving average. This increase has to be either the last determination in the patient follow-up, or maintained for at least 28 days [1]. Based on this rule and the above chosen parameters, the average median number of measurements per subject across 80 subjects is 7 (ranges from 2 to 13 ).

The models were implemented in R. We generated 5000 Gibbs samples with a burn-in of 2000 iterations. N(0, 100) priors were used for the mean parameters except that μ_τ ~ N(4, 100). Gamma(0.01, 0.01) priors were used for the precision parameters, and a noninformative prior, Beta(0.5, 0.5), was used for P_k (k = 1, · · · ,K). The posterior estimates from the reversible jump procedure and pseudo-prior approach were very similar; therefore we only reported the estimation from the reversible jump procedure. To improve the computation speed, a table lookup method was used in the indexing operation, and vectorized computation was used wherever possible. Table 1 shows the true values of the parameters (True), estimated mean (Mean), standard deviation (SD), square root of mean square error (SqrMSE), and 95% coverage probability (CP), based on 1000 simulated trials. These results indicate that the Mixture model performed well for the estimation of all parameters. In Appendix D, we also present simulation results for data simulated from a different set of parameters, which resemble the real data example, and the model also performed well as shown in Table 6.

Table 1.

Parameter estimation and performance statistics using the Mixture model based on 1000 simulated datasets.

Parameter	Arm	True	Mean	SD	SqrtMSE	CP
α		7	6.99	0.28	0.28	89
P	I	0.6	0.56	0.07	0.08	96
	II	0.8	0.79	0.07	0.07	95
β1	I	−0.8	−0.85	0.10	0.11	94
	II	−1.2	−1.21	0.08	0.08	94
μτ	I	3	3.18	0.33	0.38	93
	II	5	5.03	0.28	0.28	94
β2	I	0.4	0.39	0.03	0.03	92
	II	0.2	0.20	0.03	0.03	94

Table 6.

Parameter estimation and performance statistics using the Mixture model based on 1000 simulated datasets.

Parameter	Arm	True	Mean	SD	SqrtMSE	CP
α		7	6.97	0.31	0.31	85
P	I	0.4	0.42	0.06	0.07	97
	II	0.6	0.55	0.07	0.08	97
β1	I	−0.5	−0.49	0.05	0.05	98
	II	−0.75	−0.75	0.09	0.09	97
μτ	I	4	3.96	0.45	0.46	95
	II	2	2.21	0.24	0.32	92
β2	I	0.35	0.35	0.03	0.03	90
	II	0.35	0.34	0.03	0.03	90

3.2. Performance of the tumor growth inhibition model

In the literature, models to estimate longitudinal efficacy biomarkers following treatment are very limited. Dynamic models are the only models used to estimate longitudinal data (specifically, tumor size data) following treatment. These models assume that a non-linear tumor growth profile is a result of two latent processes: a cell growing process and a cell killing process induced by drug exposure. Since PSA can be considered as a surrogate of tumor size, dynamic models can serve as alternative models for longitudinal PSA data. Among all dynamic models in the literature [24–28], the most commonly used model was proposed by Claret et al. [26–28]. It is referred to as the tumor-growth inhibition (TGI) model. The TGI model is described by the differential equation,

$$\frac{d{y}_i(t)}{dt}=K_{L_i}{y}_i(t)-K_{D{0}_i}exp(-{\lambda }_{i}t)y_i(t)y_i(0)=y_{0i}$$

where y_i(t) is the PSA measurement at time t for subject i, and y_i(t) is described as a result of two latent processes: a growing process with a rate of K_Li and a killing process with a rate of K_D0i exp(−λ_it), which decreases exponentially with time (according to λ_i) from an initial rate of K_D0i [26].

This differential equation can be shown (details in Appendix C) to be equivalent to

$$log(y_i(t))=log(y_i(0))+K_{L_i}t+\frac{K_{D{0}_i}}{{\lambda }_i}(exp(-{\lambda }_{i}t)-1)$$

where lognormal distributions are assumed for K_Li,K_D0i and λ_i as specified in [26].

It is advisable to assume that the baseline data is also measured with error. We, therefore, modified the above model by replacing log(y_i(0)) with a random parameter a_i (i = 1, · · · , n), which can be thought as unobserved true values at baseline. This model is denoted as TGI(a).

Following Claret et al. [26], the period of PSA decline (τ_i) can be calculated as

$${\tau }_i=\frac{log(K_{D{0}_i})-log(K_{L_i})}{{\lambda }_i}$$

It is noteworthy that τ_i is negative when K_D0i < K_Li . Therefore, the TGI model provides an estimate of τ_i for all trajectories including the one without a decline phase. In contrast, the Mixture model provides an estimate of τ_i only when z_i = 1.

We used the same simulation strategy to investigate the performance of the TGI and TGI(a) model. Again, true parameter values were chosen to mimic the real data example, given by

• a_i ~ N(7, 2²) and σ² = 0.4²

• Arm I: λ_i ~ LN(−0.3, 0.5²), K_D0i ~ LN(−0.4, 0.5²), K_Li ~ LN(−1.2, 0.5²). Arm II: λ_i ~ LN(−0.5, 0.5²), K_D0i ~ LN(0.4, 0.5²), K_Li ~ LN(−1.5, 0.5²). LN denotes the lognormal distribution.

The distribution of a_i is the same as in the Mixture model. Following the calculation in [26], we estimated the posterior median (i.e., exp(a) in LN(a, b)) for each parameter. Based on the choices of the above parameters, the average median number of measurements per subject across 80 subjects is 6 (ranges from 2 to 13). The models were implemented in WinBUGS 1.4. N(0, 100) priors were used for the mean parameters and Gamma(0.01, 0.01) priors were used for the precision parameters. Both models provided accurate estimates for model parameters except λ, and the bias of λ was large especially in the TGI model (see Table 5 in Appendix D). Since the TGI(a) was better from the simulation study and also coincides with the generated data (i.e, the TGI(a) model considers that baseline data are generated with errors), we compared the Mixture model to the TGI(a) model in the next section.

3.3. Comparison of the Mixture model and the TGI(a) model

We fitted the Mixture model and TGI(a) model to the data generated in Section 3.1 (under the Mixture model) and the data generated in Section 3.2 (under the TGI(a) model), which resulted in 4 analyses (a combination of 2 models and 2 datasets). We used three Bayesian model comparison criteria to compare the Mixture model and the TGI(a) model: 1) a modified Deviance Information Criterion (DIC₃) [29], a Watanabe-Akaike information criterion (WAIC) [30] and a log-pseudo marginal likelihood (LPML) [31]. DIC₃ is preferred in our setting over the standard DIC proposed by Spiegel-halter et al. [32] since it correctly reflects the effective number of parameters in mixture models, 2) LPML is a cross-validated leave-one-out measure of a models ability to predict the data. It is valid for small and large samples and does not suffer from a heuristic justification based on large sample normality, and 3) WAIC was proposed recently and can be viewed as an improvement on the standard DIC and it also approximates Bayesian cross-validation [33]. The best model should have the smallest DIC₃ and WAIC and largest LPML.

Not surprisingly, on average TGI(a) model was the best for the data that were simulated from the TGI(a) model and the Mixture model was the best for the data that were simulated from the Mixture model, as shown in Table 2 (best diagnostic statistics are highlighted). But it appears that the Mixture model was more robust than the TGI(a) model (the differences in the diagnostic statistics were between 27 and 64 for data from the Mixture model and between 190 and 385 for data from the TGI(a) model). It is also important to note that, for data that are simulated from the TGI(a), the Mixture model was also chosen to be the best model 29–34% of the times.

Table 2.

Model comparisons

	Data from Mixture			Data from TGI(a)
Models	DIC3	WAIC	LPML	DIC3	WAIC	LPML
Mixture	710	778	−440	754	744	−449
TGI(a)	1095	1162	−630	690	711	−422
% times Mixture is better	100	100	100	30	29	34

4. Application

We applied the Mixture model, TGI and TGI(a) model to two multi-center trials for patients with advanced (metastatic) prostate cancer [1]. An experimental retinoic acid metabolism-blocking agent, liarozole, was compared with the antiandrogenic drugs, cyproterone acetate and flutamide. As in previous analyses of these data [1], we combined data from the two trials and make no distinction between the antiandrogenic drugs. Patients in both trials were in relapse after first-line endocrine therapy. We included 485 patients in our analyses, 249 treated with liarozole and 236 treated with antiandrogenic drugs. Assessments of PSA were undertaken before the start of treatment, at 2 weeks, monthly for 6 months, at 3-month intervals until treatment discontinuation or death. The number of PSA measurements per subject ranges from 2 to 19 (median is 6).

All priors were the same as in the simulation study. Based on the prior knowledge that PSA progression occurred within 6 months and the assumption that changepoint occurs about 2 months earlier than the progression time, we selected 4 as the mean for the normal prior for μ_τ (i.e.,μ_τ ~ N(4, 100)). These priors were quite vague relative to the likelihood. A sensitivity analysis to investigate the effect of the prior distributions on the parameter estimates was performed. The effect on the final estimates of increasing the variance of the prior distribution was assessed. For example, increasing the variance of the prior normal distribution (e.g., ${\sigma }_a^2,{\sigma }_{b_{1k}}^2,{\sigma }_{b_{2k}}^2$, and ${\sigma }_{\tau }^2$) from 100 to 1000 had a negligible effect on the posterior distribution of their estimates. This indicates that the priors employed were sufficiently vague relative to the likelihood, so that the prior had minimal effect on the posterior estimates.

We generated 30,000 Gibbs samples with a burn-in of 10,000 iterations, and we then used 2,000 iterations obtained from every 10th iteration for computing all posterior estimates, including the posterior mean and the highest probability density (HPD) intervals. It took about six hours for the PSA data to execute on an Intel Xeon 3.10 GHz 4GB RAM, x64 Linux computer. The MCMC chains mixed well and the convergence of the MCMC sampling algorithm was further checked using several diagnostic procedures as recommended by Cowles et al. [34].

In the first step, we verified the assumption of a constant variance by calculating a 95% HPD interval for the parameter r, as defined in section 2.4. The interval covered zero, suggesting r was not significantly different from zero and a constant variance was assumed in the analysis below.

Table 3 suggests that both the Mixture and TGI(a) models are reasonable choices for the fit of PSA data. Specifically, the Mixture model is preferred for the Liarozole arm and the TGI(a) is preferred for the Antiandrogens arm, and the TGI model seems to be the worse choice for both arms.

Table 3.

Model comparisons

Models	Antiandrogens			Liarozole
	DIC3	LPML	WAIC	DIC3	LPML	WAIC
Mixture	1758	−1125	1998	1771	−1086	1950
TGI(a)	1697	−1070	1884	1842	−1124	2021
TGI	2020	−1192	2179	1965	−1140	2104

Posterior estimates from the Mixture and TGI(a) models are displayed in Table 4. This Table shows that both models provided very close estimates for α and σ². A 95% HPD interval was calculated for the difference in each parameter between Liarozole and Antiandrogens (denoted by Difference in the Table); if the interval does not cover zero for a parameter, the two arms are considered to be significantly different for that parameter (denoted by *). Both models show that Liarozole was more effective. In the Mixture model, while the response rate in Liarozole was similar to Antiandrogens (≈ 40%), patients treated with Liarozole responded to therapy significantly longer than those treated with antiandrogenic drugs, which led to a significantly prolonged TTP, greatly increased cancer cell killing fraction as well as a longer tumor growth delay. In the TGI(a) model, the initial drug killing rate was higher in Liarozole and the tumor growth rate was also marginally higher in Liarozole. The estimates using the TGI model were similar to the TGI(a) model except that K_L is not significant in the TGI model.

Table 4.

Posterior mean and 95% HPD intervals in prostate cancer trials using the Mixture model and TGI(a) model.

Common Parameters
Mixture	σ	0.46	(0.45,0.48)
TGI(a)		0.46	(0.45,0.48)
Mixture	α	7.10	(6.91,7.28)
TGI(a)		7.04	(6.84,7.21)
Different Parameters
		Antiandrogens	Liarozole	Difference
Mixture	P	0.40(0.33,0.48)	0.43(0.35,0.50)	(−0.08,0.13)
	μτ	2.02(1.57,2.46)	4.09(3.28,4.96)	(1.12,3.07)*
	β1	−0.76(−0.97, −0.57)	−0.74(−0.90, −0.59)	(−0.25,0.27)
	β2	0.37(0.33,0.42)	0.35(0.32,0.39)	(−0.07,0.04)
	TTP	2.39(2.15,2.67)	3.41(2.94,3.87)	(0.46,1.53)*
	KF	0.46(0.38,0.55)	0.73(0.63,0.82))	(0.13,0.39)*
	TGD	2.50(1.90,3.18)	5.30(3.95,6.81)	(0.28,4.48)*
TGI(a)	λ	0.97(0.42,1.73)	0.82(0.39,1.41)	(−1.24,0.79)
	KD0	0.26(0.09,0.42)	0.62(0.40,0.97)	(0.08,0.79)*
	KL	0.27(0.22,0.32)	0.34(0.29,0.40)	(0.00,0.15)*

Figure 2 displays the estimated PSA trajectories on the observed data for 9 selected subjects. All three models fit the data reasonably well. The Mixture model was noticeably better for subject G and slightly worse for subject I. Based on the Mixture model, each fitted curve was estimated by a weighted average of a linear function and a PWL function. The weight (mixture probability) is shown on top of each panel, which indicates how likely it is that the subject responds to therapy. For instance, patient A is a responder, patient D is a non-responder, and patient F is a responder with a 48% probability. It is also important to note that the fitted curve is not limited to a linear or PWL function; for instance, patient A, B and C had a quadratic fitted curve and patient F and I had a fitted curve with a slower increasing linear function followed by a rapidly increasing linear function. These 9 subjects represent different patterns of PSA trajectories and they were selected for illustrative purposes. We want to emphasize that the model fit all 485 subjects fairly well (plots are available upon request). It is worth mentioning that profiles with two data points (such as patient H) should be interpreted with caution. If the two data points are far apart in time, it is unknown if there was a PSA decline between the two time points. In this study, there are only a few such subjects and the two time points were reasonably close, otherwise, these subjects should be excluded before using the Mixture model.

Figure 2.

log₂ PSA profiles for 9 selected patients in the prostate cancer trial. The horizontal axis is the months at which PSA are measured. Each plot represents one patient denoted by A,B,C, · · · , I. The circles are the observed data. The percentage on the top of each panel represents a mixture probability, i.e., the proportion of times that the trajectory is classified as a PWL function.

Lastly, we assessed the goodness-of-fit of each model using a quantitative measure based on posterior predictive checks. The predicted values were obtained from the posterior predictive distribution for each posterior sample from the MCMC algorithm. Figure 3 suggests that the Mixture model fit the PSA data better than the TGI and TGI(a) models. In the TGI(a) model, the observed 25% quartile and median were above the 95% confidence intervals of predictive distribution of the model; in the TGI model, the observed 75% quartile is below the 95% confidence interval. In contrast, the observed median and quartiles (25% and 75% quartiles) for the Mixture model were all well within the 95% confidence intervals.

Figure 3.

95% confidence intervals of predicted log₂ PSA reduction at the last measurement relative to baseline (a), or relative to the second measurement (b), compared with observed values (vertical lines).

5. Discussion

In this article, we extended existing mixture models to estimate non-linear PSA trajectories following treatment by introducing a mixture structure consisting of a linear function and a PWL function. These two functions have an important clinical implication: a responder is described by a PWL function and a non-responder is described by a linear function. The model is able to classify each subject as a responder or non-responder with a certain probability. Using this probability as a weight, the weighted average of the two trajectories sufficiently characterizes a variety of patterns of PSA trajectories. Furthermore, this mixture structure enables us to derive clinically useful endpoints such as a response rate and time-to- progression, as well as biologically meaningful endpoints such as a cancer cell killing fraction and tumor growth delay.

A major limitation of dynamic models is its inability to classify the PSA trajectories; therefore, it provides an estimate of the PSA decline period even when this period does not exist. We illustrated the advantage of our model through simulation studies and an actual clinical trial data, and conclude that it is an attractive alternative to dynamic models for longitudinal PSA data following treatment. The proposed mixture model is generalizable to many other diseases when longitudinal efficacy biomarker data are available (for example, CD125 in ovarian cancer and circulating tumor cells in various solid tumors). Additionally, this model allows the inclusion of covariates for the estimation of the changepoint, log₂ PSA value at the changepoint, and log₂ PSA rates of change before and after the changepoint. For instance, the mean of τ_i, μ_τ can be modelled as a linear function of the treatment groups and patient-level covariates.

The model can also incorporate data that fall below the limit of detection. Such data are not observed in the prostate cancer trials, but they are likely to occur in most clinical trials. For example, subjects with complete response have data that are below the limit of detection. Such data can be considered left-censored and their values are less than a threshold. In the posterior sampling, these data can be sampled from a truncated normal distribution as illustrated in [13].

In this article we used partial follow-up data on PSA for patients who dropped out of the study before PSA progression by assuming uninformative drop-out. Appropriate adjustment is needed if the drop-out is informative of the PSA trajectory, and the inclusion of some time-dependent covariates may potentially be useful for this adjustment.

APPENDIX

A: Posterior Computation

The Markov chain Monte Carlo procedure for estimating the posterior distributions was implemented by repeatedly drawing samples from the full conditional distributions of the parameters. The full conditional distributions for all parameters except τ_i and z_i are straightforward to compute due to the conjugacy, assuming posterior distributions are normal for the mean parameters and gamma for the precision parameters.

Sampling τ_i

The likelihood function is only continuous but not differentiable in τ_i. Skates et al. [14] and Pauler et al. [7] propose using an approximation of the likelihood to obtain a proposal density in the Metroplis-Hastings algorithm. We found that the slice sampling algorithm developed by Neal [17] is more efficient in sampling the changepoint τ_i. In the slice sampling algorithm, the lower limit is zero and the slice width is 0.2.

Sampling z_i

Updating the class indicator z_i of whether or not there is a treatment responding period is not straightforward, because different values of z_i imply different parameter spaces. If z_i = 0, two parameters (a_i, b_2i) will need to be updated; if z_i = 1, four parameters (a_i, b_1i, τ_i, b_2i) will need to be updated. Hence sampling from the full conditional distribution of z_i requires an algorithm that can move between z_i = 0 and z_i = 1. We applied two procedures that allow such moving between models, the reversible jump procedure of Green [19] and the pseudo-prior method of Carlin et al. [18] . In both procedures, we took $p(z_i=1)=p(z_i=0)=\frac{1}{2}$ as a priori. In the reversible jump procedure, prior densities of τ_i and b_1i were chosen as their proposal densities to propose a move to z_i = 1. Specifically, we generated a candidate $b_{1i}^{\ast }~N({\beta }_{1k},{\sigma }_{b_{1k}}^2)\mathrm{I}(b_{1i}<0)$ and a candidate ${\tau }_i^{\ast }~N({\mu }_{\tau k},{\sigma }_{\tau k}^2)\mathrm{I}({\mu }_{\tau k}>0)$, and the move was accepted with probability

$$min\{\text{likehood}\text{ratio}\times \text{prior}\text{ratio}\times \text{proposal}\text{ratio}\times \text{Jacobian},1\},$$

since the proposal densities are their prior distributions, prior ratio × proposal ratio = 1, and the Jacobian was also equal to 1 because $b_{1i}^{\ast }$ and ${\tau }_i^{\ast }$ were generated directly from proposal distributions. Therefore, the acceptance probability reduced to the minimum of likelihood ratio and 1,

$$min\left\{\frac{l({\mathbf{y}}_{\mathbf{i}}\mid a_i,b_{1i}^{\ast },{\tau }_i^{\ast },b_{2i},{\sigma }^2)}{l({\mathbf{y}}_{\mathbf{i}}\mid a_i,b_{2i},{\sigma }^2)},1\right\}$$

where l(.) is the likelihood of longitudinal data of subject i (i.e., y_i = (y_i1, · · · , y_ini ). The acceptance probability for a proposed move from z_i = 1 to z_i = 0 is the inverse of the above probability.

The prior densities of τ_i and b_1i were also chosen to be their pseudo-prior densities. As the name suggests, a pseudo-prior is not really a prior but only a conveniently chosen linking density, required to define completely the joint model specification. In other words, we augmented the trajectory under z_i = 0 by defining a probability distribution for a hypothetical b_1i and τ_i. These variables have no meaningful interpretation under z_i = 0, and they are only introduced to match the parameter dimensions. The full conditional distributions of b_1i and τ_i are

$$f(b_{1i},{\tau }_i\mid z_i,{\mathbf{y}}_{\mathbf{i}}\cdots )=\{\begin{array}{ll}l({\mathbf{y}}_{\mathbf{i}}\mid b_{1i},{\tau }_i,z_i=1)g(b_{1i},{\tau }_i\mid z_i=1)\hfill & \text{if}{z}_i=1;\hfill \\ g(b_{1i}^{\ast },{\tau }_i^{\ast }\mid z_i=0)\hfill & \text{if}{z}_i=0.\hfill \end{array}$$

Where $g(b_{1i}^{\ast },{\tau }_i^{\ast }\mid z_i=0)=N({\beta }_{1k},{\sigma }_{b_{1k}}^2)\mathrm{I}(b_{1i}\le 0)\times N({\tau }_0,{\sigma }_{{\tau }_0}^2)\mathrm{I}({\mu }_{\tau k}>0)$. When z_i = 1, we generate $b_{1i}^{\ast }$ and ${\tau }_i^{\ast }$ from the usual full conditional given data; when z_i = 0, we generate $b_{1i}^{\ast }$ and ${\tau }_i^{\ast }$ from their pseudo-prior densities $g(b_{1i}^{\ast },{\tau }_i^{\ast }\mid z_i=0)$. In this case when the pseudo-priors are their prior densities, we have

$$p(z_i=1)=\frac{l({\mathbf{y}}_{\mathbf{i}}\mid a_i,b_{1i}^{\ast },{\tau }_i^{\ast },b_{2i},{\sigma }^2)}{l({\mathbf{y}}_{\mathbf{i}}\mid a_i,b_{1i}^{\ast },{\tau }_i^{\ast },b_{2i},{\sigma }^2)+l({\mathbf{y}}_{\mathbf{i}}\mid a_i,b_{2i},{\sigma }^2)},$$

Hence the pseudo-prior method generates z_i as a discrete random variable of 0 or 1, in contrast to the Metropolis step in the reversible jump procedure.

B: Sampling the Non-constant Variance

In section 2.4, we modelled the variance as a function of the observed y_ij , i.e.,

$${\epsilon }_{ij}~N(0,{\sigma }_{ij}^2),$$

The full conditional distributions of σ² and r are

$$[1/{\sigma }^2\mid \cdots ]=\text{Gamma}\left(0.01+\frac{{\sum }_{i=1}^N{n}_i}{2},0.01+\frac{1}{2}\sum _{i=1}^N\sum _{j=1}^{n_i}{(v_{ij}/v_{i1})}^{2r}{(y_{ij}-{\mu }_{z,ij})}^2\right)$$

We sampled r by means of the adaptive Metropolis-Hastings algorithm [35]. The normal proposal density was centered at the previous value, and the variance in the proposal was ”refined” by using the empirical covariance from an extended burn-in period.

C: Implementation of the TGI model

$$f(r\mid \cdots )\propto exp\left\{r\sum _{i=1}^N\sum _{j=1}^{n_i}log(v_{ij}/v_{i1})-\frac{1}{2{\sigma }^2}\sum _{i=1}^N\sum _{j=1}^{n_i}{(v_{ij}/v_{i1})}^{2r}{(y_{ij}-{\mu }_{z_i,ij})}^2\right\}$$

Therefore,

$$\begin{gathered} \frac{d{y}_i(t)}{dt}=K_{L_i}{y}_i(t)-K_{D{0}_i}exp(-{\lambda }_{i}t)y_i(t)\text{and}{y}_i(0)=y_{0i} \\ \frac{d{y}_i(t)}{dt}\frac{1}{y_i(t)}=K_{L_i}-K_{D{0}_i}exp(-{\lambda }_{i}t) \\ \frac{\mathit{dlog}(y_i(t))}{dt}=K_{L_i}-K_{D{0}_i}exp(-{\lambda }_{i}t) \\ \mathit{log}(y_i(t))=K_{L_i}t+\frac{K_{D{0}_i}}{{\lambda }_i}exp(-{\lambda }_{i}t)+\mathrm{c} \\ \mathit{log}(y_i(0))=\frac{K_{D{0}_i}}{{\lambda }_i}+\mathrm{c}\Rightarrow \mathrm{c}=\mathit{log}(y_i(0))-\frac{K_{D{0}_i}}{{\lambda }_i} \end{gathered}$$

Therefore,

$$\mathit{log}(y_i(t))=\mathit{log}(y_i(0))+K_{L_i}t+\frac{K_{D{0}_i}}{{\lambda }_i}(exp(-{\lambda }_{i}t)-1)$$

In the TGI model, K_Li,K_D0i and λ_i were assumed to be lognormally distributed.

D: Additional Simulation Results

Simulation results using the TGI and TGI(a) models are presented in Table 5. Data were simulated from true parameters below, and simulation results from the Mixture model are shown in Table 6.

Table 5.

Parameter estimation and performance statistics using TGI and TGI(a) models based on 1000 simulated datasets.

Parameter	Arm	True	baseline	Mean	SD	SqrMSE	CP
α		7	TGI(a)	6.99	0.22	0.22	95
λ	I	0.74	TGI(a)	1.96	10.99	11.06	93
			TGI	6.43	86.86	87.04	85
	II	0.61	TGI(a)	0.67	0.15	0.16	93
			TGI	0.67	0.15	0.17	91
KD0	I	0.67	TGI(a)	0.99	3.33	3.34	96
			TGI	2.79	32.18	32.25	92
	II	1.49	TGI(a)	1.54	0.22	0.23	94
			TGI	1.54	0.22	0.22	93
KL	I	0.30	TGI(a)	0.29	0.07	0.07	90
			TGI	0.29	0.06	0.00	90
	II	0.22	TGI(a)	0.21	0.04	0.03	92
			TGI	0.22	0.03	0.00	92

• Arm I: P₁ = 0.4, b_1i ~ N(−0.5, 0.1²), τ_i ~ N(4, 1.5²), b_2i ~ N(0.35, 0.1²). Arm II: P₂ = 0.6, b_1i ~ N(−0.75, 0.16²), τ_i ~ N(2, 1²), b_2i ~ N(0.35, 0.1²).