Joint modelling of longitudinal and multi-state processes: application to clinical progressions in prostate cancer

Abstract

Joint modelling of longitudinal and survival data is increasingly used in clinical trials on cancer. In prostate cancer for example, these models permit to account for the link between longitudinal measures of prostate-specific antigen (PSA) and time of clinical recurrence when studying the risk of relapse. In practice, multiple types of relapse may occur successively. Distinguishing these transitions between health states would allow to evaluate, for example, how PSA trajectory and classical covariates impact the risk of dying after a distant recurrence post-radiotherapy, or to predict the risk of one specific type of clinical recurrence post-radiotherapy, from the PSA history. In this context, we present a joint model for a longitudinal process and a multi-state process which is divided into two sub-models: a linear mixed sub-model for longitudinal data, and a multi-state sub-model with proportional hazards for transition times, both linked by a function of shared random effects. Parameters of this joint multi-state model are estimated within the maximum likelihood framework using an EM algorithm coupled with a quasi-Newton algorithm in case of slow convergence. It is implemented under R, by combining and extending mstate and JM packages. The estimation program is validated by simulations and applied on pooled data from two cohorts of men with localized prostate cancer. Thanks to the classical covariates available at baseline and the repeated PSA measurements, we are able to assess the biomarker’s trajectory, define the risks of transitions between health states, and quantify the impact of the PSA dynamics on each transition intensity.

1. Introduction

In longitudinal health studies, marker data are usually collected at repeated measurement times until the occurrence of an event such as disease relapse or death, with the objective to study the link between these two correlated processes or use the information brought by the marker’s dynamics to explain or predict the time to event. In such analyses, the repeated measurements of the marker should not be considered as a standard time-dependent covariate in a survival model [1, 2] because the marker is an internal outcome measured with error and at discrete times whereas the Cox model assumes that the exact values of the explanatory variables are known for all the individuals at risk at each event time. To counteract these weaknesses, the two processes can be modelled jointly [3, 4]. The principle is to define two sub-models (one mixed sub-model for the longitudinal data and one survival sub-model for the time-to-event data) and use a common latent structure to link them. The shared random effect models, notably developed by Tsiatis and Davidian [5], are the most popular joint models. They usually assume that a function of the random effects from the linear mixed model is included as covariate in the survival model. This function can be any underlying features of the marker dynamics.

The joint modelling method is very useful in prostate cancer. The prostate-specific antigen (PSA), which is a protein secreted by the prostate, is found to be over-expressed in the presence of prostate cancer. This blood-based longitudinal tumour marker is commonly used by clinicians to monitor patients with localized prostate cancer following treatment (radiation therapy or surgery) in order to detect subclinical presence of disease. Proust- Lima et al. [6]; Taylor et al. [7] and Yu et al. [8] showed, through various types of joint models, that the dynamics of this biomarker, along with the pretreatment PSA level and other factors measuring the aggressiveness of cancer cells and the extent of the tumour, were risk factors for progression and permitted one to dynamically predict (i.e. using PSA to adapt prediction over time) the risk of clinical relapse.

In practice, a patient may experience a succession of clinical progression events with for example a local recurrence, followed by a distant metastatic recurrence and then death. So, instead of the occurrence of a single clinical event, the progression of prostate cancer should be defined as a multi-state process with a focus on the transitions between clinical states and the impact of the biomarker dynamics on it. This is essential to understand and predict accurately the course of the disease, and it is of particular relevance for the clinicians that need to distinguish the different types of events in order to properly adapt the treatment.

Some authors already extended the classical joint modelling framework to multiple time-to-event data. Chi and Ibrahim [9] proposed a joint model for multivariate longitudinal data and multivariate survival data. Liu and Huang [10] and Kim et al. [11] looked into the simultaneous study of three correlated outcomes: longitudinal data, times of recurrent events and time of terminal event. Elashoff et al. [12] and Rizopoulos [13] extended the joint model to competing risks data, which allows to characterize the cause of survival event. Dantan et al. [14] developed a joint model with latent state for longitudinal data and illness-death data. Tom and Farewell [15] proposed a complex multi-state model that combined an intermittently observed longitudinal categorical process and a multi-state process. Recently, Andrinopoulou et al. [16] studied simultaneously two longitudinal markers and competing events. However, the joint study of Gaussian longitudinal data and multi-state data has never been proposed and implemented. Thus, we introduce a joint model with shared random effects for repeated measurements of a longitudinal marker and times of transitions between multiple states. It consists in a linear mixed model and a multi-state model with transition-specific proportional intensities, both linked by shared random effects.

The computational aspect is the main obstacle in the development of joint models with shared random effects. As explained by Gould et al. [17], the R package JM, developed by Rizopoulos [18], has enabled many advances in the use of joint modelling, particularly through efficient numerical integrations. On the other hand, the R package mstate, developed by De Wreede et al. [19], provides estimation of multi-state models. In the present work, we combine and adapt these two packages in order to estimate joint multi-state models. Thus, the implementation is easy and effective. Through the adaptation of jointModel() function of JM package, our approach uses the maximum likelihood approach, which is performed using the EM algorithm coupled with a quasi-Newton algorithm in case of slow convergence. The software advantage is that it keeps the features, syntax and outputs of JM.

The paper is organized as follows. Section 2 presents the joint model for longitudinal and multi-state processes. Estimation and implementation procedures are detailed in Section 3 and validated by simulations in Section 4. The model is applied to two cohorts of men with prostate cancer in Section 5 and a brief discussion is finally given in Section 6.

2. Joint multi-state model

2.1. Notations

For each individual i, a longitudinal process and a multi-state process are observed. Let {E_i(t), t ≥ 0} be the multi-state process where E_i(t) denotes the occupied state by subject i at time t and takes values in the finite state space S = {0, 1, …, M}. It is assumed that the multi-state process is continuous and observed between the left truncation time (time of entry in the study) T_i0 and the right censoring time C_i, so that the observed process is E_i = {E_i(t), T_i0 ≤ t ≤ C_i}. We further consider that E_i is a non-homogeneous Markov process. The Markov property ensures that the future of the process depends only on the present state and not on the past state, i.e. Pr (E_i(t + u) = k|E_i(t) = h, {E_i(s), s < t}) = Pr (E_i(t + u) = k|E_i(t) = h), ∀h, k ∈ S, ∀u ≥ 0 [19], and the non-homogeneous property guarantees that the time since T_i0 impacts the future evolution of the process. Let us consider $T_i={(T_{i1},T_{i2},\dots ,T_{i{m}_i})}^{\mathrm{\top }}$ the vector of the m _i ≥ 1 observed time(s) for individual i, with T_ir < T_i(r+1), ∀r ∈ {0, …, m_i − 1}, and where ^⊤ denotes the transpose operator. If the last observed state for subject i (E_i(T_imi)) is absorbing, that is it is impossible to leave it once entered (typically death), we observe m_i direct transition(s). Otherwise, $T_{i{m}_i}$ equals C_i the right censoring time and we observe m_i − 1 direct transition(s). We define by ${\delta }_i={\left({\delta }_{i1},\dots ,{\delta }_{i{m}_i}\right)}^{\mathrm{\top }}$ the vector of observed transition indicator(s), with δ_i(r+1) equals 1 if a direct transition is observed at time T_i(r+1) (i.e. E_i(T_ir) ≠ E_i(T_i(r+1))) and 0 otherwise, ∀r ∈ {0, …, m_i − 1}. For each subject i, we also observe $Y_i={\left(Y_{i1},\dots ,Y_{i{n}_i}\right)}^{\mathrm{\top }}$ the vector of n_i measures of the marker collected at times $t_{i1},\dots ,t_{i{n}_i},$ with $t_{i{n}_i}\le T_{i{m}_i}$.

2.2. Joint multi-state model formulation

The joint multi-state model is decomposed into two sub-models: a linear mixed sub-model for the longitudinal data (repeated measurements of the biomarker) and a multi-state model with transition-specific proportional intensities for the event history data (transition and censoring times), both linked by a function of the shared random effects.

2.2.1. Longitudinal sub-model

To model the trajectory of the longitudinal marker, we use a linear mixed model. Under Gaussian assumptions, we assume that Y_ij the observed measure of the marker at time point t_ij is a noisy measure of the true level $Y_i^{\ast }\left(t_{ij}\right)$. This non-observed level $Y_i^{\ast }\left(t_{ij}\right)$ is explained according to time and covariates with fixed effects β at the population level, and random effects b_i that take into account the correlation between repeated measures of the same individual:

$$\begin{array}{c}{Y}_{ij}=Y_i^{\ast }\left(t_{ij}\right)+{\epsilon }_{ij}\\ =X_i^L{\left(t_{ij}\right)}^{\mathrm{\top }}\beta +Z_i{\left(t_{ij}\right)}^{\mathrm{\top }}{b}_i+{\epsilon }_{ij},\end{array}$$ (1)

with $X_i^L\left(t_{ij}\right)$ and Z_i(t_ij) the vectors of possibly time-dependent covariates associated with the p-vector of fixed effects β and the q-vector of random effects $b_i,b_i~\mathcal{N}\left(0,D\right)$, respectively. Note that ${\epsilon }_i={\left({\epsilon }_{i1},\dots ,{\epsilon }_{i{n}_i}\right)}^{\mathrm{\top }}~\mathcal{N}\left(0,{\sigma }^2{I}_{n_i}\right)$ where I is the identity matrix; ɛ_i and b_i are independent.

2.2.2. Multi-state sub-model

To model the transition times, we use a Markov multi-state model with proportional hazards that takes into account the marker’s dynamics through the shared random effects b_i. Thus, for the transition from state h ∈ S to state k ∈ S, the transition intensity at time t takes the form:

$$\begin{array}{c}{\lambda }_{\mathit{hk}}^i(t|b_i)=\underset{\mathrm{d}t\to 0}{\text{lim}}\frac{\text{Pr}\left(E_i\left(t+\mathrm{d}t\right)=k|E_i\left(t\right)=h;b_i\right)}{\mathrm{d}t}\\ ={\lambda }_{hk,0}\left(t\right)\text{exp}\{X_{hk,i}^{S\mathrm{\top }}{\gamma }_{hk}+W_{hk,i}{\left(b_i,t\right)}^{\mathrm{\top }}{\eta }_{hk}\},\end{array}$$ (2)

with λ_hk,0(.) the parametric baseline intensity (Weibull, piecewise constant or B-splines for example) and $X_{hk,i}^S$ the vector of prognostic factors associated with the r-vector of coefficients γ_hk. The multivariate function W_hk,i(b_i, t) defines the dependence structure between the longitudinal and multi-state processes. We can choose $W_{hk,i}\left(b_i,t\right)=Y_i^{\ast }\left(t\right)$ (the true current level of the marker), or $W_{hk,i}\left(b_i,t\right)=\partial Y_i^{\ast }\left(t\right)/\partial t$ (the true current slope), $W_{hk,i}\left(b_i,t\right)={\left(Y_i^{\ast }\left(t\right),\partial Y_i^{\ast }\left(t\right)/\partial t\right)}^{\mathrm{\top }}$ (both), or any other function of the random effects in the context under study. Thus, the s-vector of coefficients η_hk quantifies the impact of the longitudinal marker’s dynamics on the transition intensity between the states h and k.

3. Estimation

3.1. Likelihood

The parameters of this joint model are estimated in the maximum likelihood framework. Since the longitudinal and multi-state processes are independent conditionally on the random effects, the complete observed likelihood is obtained through the product of the individual contributions to the likelihood for the N individuals as:

$$L\left(\theta \right)={\displaystyle \prod _{i=1}^N{\displaystyle {\int }_{{\text{ℝ}}^q}{f}_Y\left(Y_i|b_i;\theta \right)f_E\left(E_i|b_i;\theta \right)f_b\left(b_i;\theta \right)\mathrm{d}{b}_i}},$$ (3)

where θ is the vector of all the parameters contained in (1) and (2), and f (.) is a probability density function.

In the longitudinal part, described by the linear mixed model (1), the conditional longitudinal outcomes are such that:

$$f_Y\left(Y_i|b_i;\theta \right)=\frac{1}{{\left(2\pi {\sigma }^2\right)}^{n_i/2}}\text{exp}\mathbf{\left(}-\frac{{\Vert Y_i-X_i^{L\mathrm{\top }}\beta -Z_i^{\mathrm{\top }}{b}_i\Vert }^2}{2{\sigma }^2}\mathbf{\right)},$$ (4)

where ‖x‖ denotes the Euclidean norm of vector x, $X_i^L$ is the matrix of covariates with row vectors $X_i^L{\left(t_{ij}\right)}^{\mathrm{\top }},j=1,\dots ,n_i$, and likewise Z_i = {Z_i(t_ij)}.

For the multi-state part, let $P_{hk}^i\left(s,t\right)$ be the transition probability from state h to state k between times s and t for individual i, i.e. $P_{hk}^i\left(s,t\right)=\text{Pr}\left(E_i\left(t\right)=k|E_i\left(s\right)=h\right)$. For each r ɛ {0, …, m_i − 1}, the continuity and Markov assumptions imply that individual i remains in state E_i(T_ir) between times T_ir and T_i(r+1) with probability $P_{E_i\left(T_{ir}\right),E_i\left(T_{ir}\right)}^i(T_{ir},T_{i\left(r+1\right)}|b_i)$, and transits to state E_i(T_i(r+1)) with intensity ${\lambda }_{E_i\left(T_{ir}\right),E_i\left(T_{i\left(r+1\right)}\right)}^i(T_{i\left(r+1\right)}|b_i)$when T_i(r+1) is an observed transition time. By conditioning on E_i(T_i0), this translates in the individual contribution to the likelihood:

$$\begin{array}{l}{f}_E(E_i|b_i;\theta )=\prod _{r=0}^{m_i-1}\mathbf{\left\{}{P}_{E_i\left(T_{\mathit{ir}}\right),E_i\left(T_{ir}\right)}^i(T_{\mathit{ir}},T_{i\left(r+1\right)}|b_i){\lambda }_{E_i\left(T_{\mathit{ir}}\right),E_i\left(T_{i\left(r+1\right)}\right)}^i{(T_{i\left(r+1\right)}|b_i)}^{{\delta }_{i\left(r+1\right)}}\right\}\\ =\prod _{r=0}^{m_i-1}\mathbf{\left\{}\text{exp}\mathbf{\left(}{\displaystyle {\int }_{T_{ir}}^{T_{i\left(r+1\right)}}{\lambda }_{E_i\left(T_{ir}\right),E_i\left(T_{ir}\right)}^i\left(u|b_i\right)\mathit{du}}\mathbf{\right)}{\lambda }_{E_i\left(T_{\mathit{ir}}\right),E_i\left(T_{i\left(r+1\right)}\right)}^i{\left(T_{i\left(r+1\right)}|b_i\right)}^{{\delta }_{i\left(r+1\right)}}\mathbf{\right\}}\end{array}$$ (5)

with ${\lambda }_{hh}^i\left(t\right)=-{\displaystyle {\sum }_{k,k\ne h}{\lambda }_{hk}^i\left(t\right)}$. The possible delayed entry is accounted for by conditioning on E_i(T_i0).

Finally, the random effects b_i follow a multivariate Gaussian distribution such that:

$$f_b\left(b_i;\theta \right)=\frac{1}{{\left(2\pi \right)}^{q/2}\text{det}{\left(D\right)}^{1/2}}\text{exp}\mathbf{\left(}-\frac{b_i^{\mathrm{\top }}{D}^{-1}{b}_i}{2}\mathbf{\right)}.$$ (6)

3.2. Implementation

The joint multi-state model has been implemented under R, via the combination of two well-known packages: mstate for multi-state models and JM for joint models with shared random effects. To fit semi-parametric Markov multi-state models, mstate prepares the database for multi-state analysis, more specifically by defining each patient’s history as a series of rows, one for each transition at risk for each individual (in contrast with only one data record (row) per individual in a classical survival analysis). By stratifying on the transition type, the standard coxph() function of the R package survival can then be used to fit transition-specific Cox models. With standard longitudinal and time-to-event data, JM package initialises the values of the parameters with function lme() ( nlme package) for the longitudinal sub-model and coxph() ( survival package) for the survival sub-model. Then, function jointModel() carries out the estimation procedure.

So by replacing the standard call to coxph() by the call to coxph() on the data prepated with mstate, an extended jointModel() function, called JMstateModel(), can carry out the estimation procedure of the joint model for longitudinal and multi-state data. The implementation procedure thus includes four steps:

• lme() function ( nlme package) to initialise the parameters of the longitudinal sub-model;

• msprep() and expand.covs() functions ( mstate package) to prepare the multi-state data;

• coxph() function ( survival package) applied to the prepared data to initialise the parameters of the multi-state sub-model;

• JMstateModel() function to estimate all the parameters of the joint multi-state model.

A detailed example is given in Web Appendix A, and full detailed examples are available on https://github.com/LoicFerrer/JMstateModel/.

3.3. Algorithm

JMstateModel() function computes and maximises the joint log-likelihood extended to handle multi-state data using integration and optimisation algorithms available in JM package. Thus, the procedure combines an EM algorithm coupled with a quasi-Newton algorithm if the convergence is not achieved. Furthermore, the integral with respect to time in (5) and the integral with respect to the random effects in (3) do not have an analytical solution. These integrals are approached by numerical integration. The integrals over time are approximated using Gauss-Kronrod quadratures, and the integral over the random effects using pseudo-adaptive Gauss-Hermite quadratures. Inference is provided by asymptotic properties for maximum likelihood estimators. The variance-covariance matrix of the parameter estimates is based on the inverse of the Hessian matrix. Details on the optimisation procedure, the EM algorithm and the numerical integrations can be found in Rizopoulos [13].

The main difficulty with the inference comes from the numerical approximation of the integral over the random effects, especially when the dimension of the random effects increases. The pseudo-adaptive Gauss-Hermite quadrature proposed by Rizopoulos [20] centers the integral using the posterior distribution of the random effects, derived from the initial linear mixed model. This reduces the required number of quadrature points compared to the standard Gaussian quadrature and avoids the intensive computations of the adaptive quadrature. We went one step further by repeating this procedure: the joint model can be estimated once using the pseudo-adaptive technique and it can then be reestimated by starting from the previously estimated parameters and centering the integral on the predicted random effects derived from the joint model rather than on the linear mixed model. We expect the integral to be more accurate while using a relatively small number of quadrature points. In the remainder, the technique is referred to as the multi-step pseudo-adaptive Gauss-Hermite rule. More details are in Web Appendix B.

4. Simulation study

The estimation procedure was validated in a simulation study.

4.1. Data generation

In one specific replicate, the longitudinal and multi-state data were generated for each subject i = 1, …, 500, according to the joint multi-state model defined as:

$$\mathbf{\{}\begin{array}{c}{Y}_{\mathit{ij}}=Y_i^{\ast }\left(t_{\mathit{ij}}\right)+{\epsilon }_{\mathit{ij}}\\ =({\beta }_0+{\beta }_{0,x}{X}_i+b_{i0})+\\ ({\beta }_1+{\beta }_{1,x}{X}_i+b_{i1})\times \left({\left(1+t_{\mathit{ij}}\right)}^{-1.2}-1\right)+\\ ({\beta }_2+{\beta }_{2,x}{X}_i+b_{i2})\times t_{\mathit{ij}}+{\epsilon }_{\mathit{ij}},\\ {\lambda }_{\mathit{hk}}^i(t|b_i)={\lambda }_{hk,0}(t)\text{exp}\{{\gamma }_{\mathit{hk}}{X}_i+{\eta }_{hk,\text{level}}{Y}_i^{\ast }\left(t\right)+{\eta }_{hk,\text{slope}}\partial Y_i^{\ast }(t)/\partial t\},\end{array}$$ (7)

where the multi-state process that included three states (h, k ∈ {0, 1, 2}) and three transitions is described in Figure 1.

Figure 1.

Simulated multi-state process. Arrows indicate the directions of the possible transitions. λ_hk(t) characterizes the intensity of transition between states h and k at time t. The matrix ${\overline{\psi }}_{\text{sim}}$ has size (3, 3) and is composed of elements ${\overline{\psi }}_{\text{sim,}\left(h+1\right)\left(k+1\right)},h,k\in \left\{0,1,2\right\}$, where ${\overline{\psi }}_{\text{sim},(h+1)\left(k+1\right)}$ is the average number of observed direct transitions h → k over the 500 replicates. The diagonal elements ${\overline{\psi }}_{\text{sim},(h+1)\left(h+1\right)}$ denote the average number of patients who were censored in state h. Note that the sum of elements of a row (h + 1) of ${\overline{\psi }}_{\text{sim}}$ corresponds to the average number of patients who experienced the state h.

The same shape of trajectory as in the application was assumed with ((1 + t)^−1.2 − 1) for a short term drop and t for a long term linear trend. First, X_i and b_i = (b_i0, b_i1, b_i2)^⊤ were generated according to normal distributions 0 with mean 2.04 and variance 0.5, and mean vector $\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)$ and variance-covariance matrix $\left(\begin{array}{ccc}0.363& 0.345& 0.011\\ 0.345& 1.742& 0.310\\ 0.011& 0.310& 0.173\end{array}\right),$ respectively. The times of measurements were t_ij = 0, 0.33, 0.67, …, 16.33, and ɛ_ij was generated from a normal distribution with mean zero and variance 0.074. The log baseline intensities were linear combinations of cubic B-splines with the same knot vector (0.004, 7.458, 18.201)^⊤ for the three transitions, and the vectors of spline coefficients (−5.537, −4.373, −4.541, −7.524, −5.205)^⊤ for transition 0 → 1, (−5.231, −4.122, −3.815, −1.495, −0.887)^⊤ for transition 0 → 2, and (−2.157, −2.491, −2.175, −0.975, −0.472)^⊤ for transition 1 → 2. Parameters values and knot locations were chosen according to the application data described in Section 5.

The procedure described in Beyersmann et al. [21] and Crowther and Lambert [22] was used to generate the vector of observed times $T_i={\left(T_{i1},\dots ,T_{i{m}_i}\right)}^{\mathrm{\top }}$. For each individual i, the censoring time C_i was generated from an uniform distribution on [1, 25], and the vector of true transition times $T_i^{\ast }={\left(T_{i,01}^{\ast },T_{i,02}^{\ast },T_{i,12}^{\ast }\right)}^{\mathrm{\top }}$ was generated according to the following procedure: (1) three random numbers u_i,01, u_i,02 and u_i,12 were generated from three independent standard uniform distributions; (2) $T_{i,01}^{\ast }$ and $T_{i,02}^{\ast }$ were generated by solving ${\int }_0^{T_{i,0k}^{\ast }}{\lambda }_{0k}^i\left({\nu }_{0k}|b_i\right)\mathrm{d}{\nu }_{0k}+\text{log}\left(u_{i,0k}\right)=0$, for k = 1, 2, through the Brent’s univariate root-finding method [23]; (3) then, the true transition time $T_{i,12}^{\ast }$ was generated by solving ${\int }_{T_{i,01}^{\ast }}^{T_{i,12}^{\ast }}{\lambda }_{12}^i({\nu }_{12}|b_i)\mathrm{d}{\nu }_{12}+\text{log}(u_{i,12})=0$. Finally, by comparing $T_i^{\ast }$ and C_i, the vector T_i, which characterizes the multi-state process, was deduced.

The longitudinal measurements, generated from the linear mixed sub-model, were truncated at T_i1 the first observed time of the multi-state process.

4.2. Estimated model

The model defined in (7) was used for the estimation with $b_i~\mathcal{N}\mathbf{\left(}\mathbf{\left(}\begin{array}{c}0\\ 0\\ 0\end{array}\mathbf{\right)},\mathbf{\left(}\begin{array}{ccc}{D}_{11}& D_{12}& D_{13}\\ D_{12}& D_{22}& D_{23}\\ D_{13}& D_{23}& D_{33}\end{array}\mathbf{\right)}\mathbf{\right)}$ and ${\epsilon }_{ij}~\mathcal{N}\left(0,{\sigma }^2\right)$. The log baseline intensities were approximated by a linear combination of cubic-splines with one internal knot placed at the median of the observed transition times.

4.3. Simulation results

The simulations results were obtained through 500 replicates of 500 individuals. Each joint multi-state model was estimated using 3 and 9 pseudo-adaptive Gauss-Hermite quadrature points, and a two-step pseudo-adaptive Gauss-Hermite quadrature using 9 quadrature points at each step. The simulation results are presented in Table 1.

Table 1.

Simulation results according to 3 and 9 quadrature points using the pseudo-adaptive Gauss-Hermite rule (called one-step), and 9–9 quadrature points using the two-step adaptive Gauss-Hermite rule (called two-step). For each scenario, the statistics are (from left to right): mean, mean standard error, standard deviation, relative bias (in percentage) and coverage rate (in percentage).

	True value	3 Gauss-Hermite quadrature points (one-step)					9 Gauss-Hermite quadrature points (one-step)					9–9 Gauss-Hermite quadrature points (two-step)
		Mean	$\overline{\text{StdErr}}$	StdDev	Rel. bias	Cov.rate	Mean	$\overline{\text{StdErr}}$	StdDev	Rel. bias	Cov. rate	Mean	$\overline{\text{StdErr}}$	StdDev	Rel. bias	Cov. rate
Longitudinal process
β0	−0.255	−0.254	0.087	0.091	−0.5	95.4	−0.253	0.088	0.091	−0.7	95.4	−0.252	0.088	0.091	−1.0	95.6
β0,X	0.799	0.797	0.040	0.043	−0.2	94.2	0.797	0.040	0.043	−0.2	94.6	0.797	0.041	0.043	−0.2	95.0
β1	0.948	0.950	0.175	0.198	0.2	91.0	0.951	0.189	0.197	0.3	94.4	0.954	0.196	0.196	0.6	95.4
β1,X	0.905	0.894	0.081	0.093	−1.2	90.4	0.902	0.087	0.093	−0.3	92.8	0.903	0.091	0.092	−0.2	94.4
β2	−0.088	−0.085	0.022	0.059	−3.7	56.0	−0.081	0.045	0.060	−7.6	83.2	−0.084	0.059	0.060	−5.0	95.2
β2,X	0.207	0.198	0.010	0.028	−4.2	49.0	0.202	0.018	0.028	−2.6	79.6	0.204	0.027	0.028	−1.4	93.0
log(σ)	−1.299	−1.300	0.007	0.008	0.0	93.2	−1.300	0.007	0.008	0.0	93.6	−1.300	0.007	0.008	0.0	93.6
Multi-state process
γ01,X	0.197	0.199	0.130	0.132	1.2	94.6	0.199	0.130	0.133	0.9	94.0	0.199	0.130	0.132	0.7	94.4
γ02,X	0.170	0.182	0.124	0.119	7.5	95.8	0.182	0.124	0.119	7.4	95.4	0.182	0.124	0.119	7.3	95.8
γ12,X	−0.243	−0.232	0.168	0.187	−4.7	91.8	−0.235	0.168	0.188	−3.5	91.4	−0.234	0.168	0.188	−3.6	92.4
η01,level	0.419	0.423	0.097	0.100	0.9	93.6	0.419	0.097	0.100	−0.1	93.2	0.418	0.097	0.100	−0.4	93.2
η02,level	−0.091	−0.098	0.052	0.056	7.1	93.2	−0.099	0.052	0.056	8.3	93.8	−0.099	0.052	0.056	8.5	93.4
η12,level	0.046	0.054	0.088	0.092	16.5	93.8	0.052	0.087	0.091	12.2	94.4	0.052	0.087	0.092	12.3	94.2
η01,slope	2.919	2.909	0.453	0.458	−0.3	94.6	2.940	0.455	0.460	0.7	94.6	2.952	0.455	0.464	1.1	94.6
η02,slope	1.142	1.183	0.457	0.458	3.6	94.0	1.197	0.458	0.461	4.8	93.0	1.199	0.458	0.461	4.9	93.8
η12,slope	0.134	0.090	0.836	0.857	−32.8	95.0	0.109	0.834	0.851	−18.4	95.0	0.113	0.834	0.852	−15.4	94.8
Random effects
D11	0.363	0.360	0.026	0.025	−0.8	95.0	0.360	0.026	0.025	−0.8	95.2	0.360	0.026	0.025	−0.8	95.0
D12	0.345	0.342	0.046	0.045	−1.0	95.4	0.342	0.046	0.045	−0.9	94.8	0.342	0.046	0.045	−0.9	95.0
D13	0.011	0.011	0.013	0.013	−0.8	95.2	0.011	0.013	0.013	−0.7	95.0	0.011	0.013	0.013	−0.3	95.4
D22	1.742	1.729	0.131	0.133	−0.8	94.4	1.732	0.132	0.133	−0.6	94.2	1.732	0.132	0.133	−0.6	94.2
D23	0.310	0.307	0.033	0.033	−0.7	94.4	0.309	0.033	0.033	−0.4	94.2	0.309	0.034	0.033	−0.3	94.4
D33	0.173	0.171	0.012	0.013	−1.0	93.0	0.171	0.013	0.013	−0.7	93.8	0.172	0.013	0.013	−0.6	94.2

These results were very satisfying with unbiased estimates and correct 95% coverage rates. They showed however the need to use a certain number of Gauss-Hermite quadrature points to approximate the integral over the random effects. Indeed, the use of 3 Gauss-Hermite quadrature points using the pseudo-adaptive Gauss-Hermite rule induced poor coverage rates of the parameters associated with the long time effect in the longitudinal sub-part. The underestimation of the variance parameters was almost corrected using 9 quadrature points in the pseudo-adaptive Gauss-Hermite rule, and finally the estimated Hessian was good with 9 and 9 quadrature points using the two-step pseudo-adaptive Gauss-Hermite rule. Overall, these results confirmed the good performances of the implemented procedure. To further investigate whether the technique could be applied to more complex multi-state data, we ran another simulation study with 5 states and 10 transitions as in the application. In this second simulation, the longitudinal part was simplified by assuming a linear trajectory over time. Although some direct transitions did not have a lot of information, the coverage rates of the multi-state model parameters were good. Results are detailed in Web Appendix C.

5. Application

We analysed data from patients with a localized prostate cancer treated by external beam radiotherapy. The analysis aimed to explore the link between PSA dynamics and transition intensities between clinical states, as well as to describe PSA repeated measurements and times of transitions between health states.

5.1. Data description

Our study focuses on 1474 men with a clinically localized prostate cancer and treated by external beam radiotherapy (EBRT): 629 patients come from the multi-center clinical trial RTOG 9406 (Radiation Therapy Oncology Group, USA) in which data collection has been conducted from 1994 to 2013 [24], and 845 patients come from the cohort of the British Columbia Cancer Agency (BCCA) in Vancouver, Canada [25] with examinations between 1994 and 2012 (Table 2). During his follow-up, a patient can possibly go through several states defined as local recurrence, distant recurrence, initiation of hormonal therapy (HT) and death, due or not to prostate cancer. The initiation of salvage hormonal therapy, which is an additional treatment prompted by physician observed signs in PSA or clinical signs, is designed to prevent growth of potentially present subclinical cancer. This intervention is not planned at diagnosis or initiated by any precise rule, but is rather based on a mutual agreement between the clinician and his patient. Thus, it is treated as a disease state transition representing failure of the initial treatment to satisfactorily control the disease. Furthermore, as recommended in Proust-Lima et al. [6], we only considered the local relapses which took place three years or later after radiation, or within three years of EBRT when the last PSA value was > 2 ng/ml. PSA data were collected at regular visits, for a median number of 10 PSA measurements per patient. Note that PSA data were collected between the end of EBRT and the occurrence of the first event (first clinical recurrence, hormonal therapy, death or censorship). Subjects with only one PSA measure were excluded, and subjects who had an event in the first year after EBRT were excluded to prevent the inclusion of patients with substantial residual initial tumors. As shown in Table 2, three baseline factors were considered: the pre-therapy level of PSA in the log scale (iPSA), the T-stage category which characterizes the tumour size (3 categories were considered: 2; 3–4 versus 1 in reference), and the Gleason score category which measures the aggressiveness of cancer cells (3 categories: 7; 8–10 versus 2–6 in reference). In the models, a cohort covariate was also considered coded as 1 for RTOG 9406 and −1 for BCCA.

Table 2.

Description of the two cohorts.

Cohort	RTOG 9406	BCCA	Pooled
Study period	1994–2013	1994–2012
Number of patients	629	845	1474
Number of PSA measures per patient	13 (4, 23)	9 (3, 15)	10 (3, 21)
iPSA∗	2.0 (1.0, 3.0)	2.1 (0.6, 3.3)	2.1 (0.8, 3.1)
Clinical T-stage
1	355 (56.4%)	184 (21.8%)	539 (36.6%)
2	261 (41.5%)	514 (60.8%)	775 (52.6%)
3–4	13 (2.1%)	147 (17.4%)	160 (10.9%)
Gleason score
2–6	424 (67.4%)	605 (71.6%)	1029 (69.8%)
7	167 (26.6%)	189 (22.4%)	356 (24.2%)
8–10	38 (6.0%)	51 (6.0%)	89 (6.0%)
Mean time of first event†	9.8 (2.3, 15.9)	7.7 (1.9, 14.1)	8.2 (2.0, 15.0)
Mean time of last contact‡	11.6 (2.9, 16.7)	9.0 (3.4, 14.8)	9.7 (3.1, 15.9)

Continuous data: Median (5th and 95th percentiles).

Categorical data: Amount (percentage).

Times are in years since the end of EBRT.

^∗Pre-therapy PSA value (ng/ml) in the log(. + 0.1) scale.

^†Minimum between the time of first transition and the time of censoring.

^‡Minimum between the time of death and the time of censoring.

The PSA individual trajectories collected between the end of EBRT and the occurrence of the first event are depicted in Figure 2. Overall, this longitudinal process is biphasic, with a decrease in the level of PSA in the first years following the end of EBRT, and a subsequent stabilisation or linear rise thereafter. According to the type of first relapse, the biomarker’s long-term increase may have different intensities (see “Hormonal Therapy” and “Censorship” for example).

Figure 2.

Individual trajectories of log (PSA + 0.1) after the end of EBRT and according to the type of first relapse in the two cohorts (N = 1474). ψ_0k is detailed in Figure 3.

The multi-state data are depicted through the transitions between the 5 states and the corresponding amount of observed direct transitions in Figure 3. From the end of EBRT (state 0), a patient can experience either a transition to a localized recurrence (state 1), an hormonal therapy (state 2), a distant recurrence (state 3) or death (absorbing state 4). After a localized recurrence (state 1), a patient may initiate a HT (state 2) or experience either a distant recurrence (state 3) or die (state 4). After initiation of HT, a patient may only experience a distant recurrence or die, and finally, after a distant recurrence, a patient may only die. In total, 144 subjects had a local recurrence; 317 men initiated an hormonal therapy including 90 after a local recurrence; 90 men had a distant recurrence including 10 directly after a local recurrence and 33 after a HT initiation. In total, 802 patients died including 523 who did not have another recorded progression of the cancer before. Among the 672 men who were censored during the follow-up, 533 were censored before experiencing any clinical progression.

Figure 3.

Multi-state representation of the clinical progressions in prostate cancer. Arrows indicate the directions of the possible transitions (N = 1474). λ_hk(t) characterizes the intensity of transition between states h and k at time t. Matrix ψ has size (5, 5) and is composed of elements ψ_(h+1)(k+1), h, k ∈ {0, 1, 2, 3, 4}, where ψ_(h+1)(k+1) is the number of observed direct transitions h → k. Diagonal elements ψ_(h+1)(h+1) denote the number of patients who were censored in state h. Note that the sum of elements of one row (h + 1) of ψ corresponds to the number of patients who entered state h.

5.2. Specification of the joint model

The joint multi-state model being a complex model, a step-by-step procedure was carried out to specify the joint model. The specifications of the longitudinal and multi-state sub-models were based on two separate analyses, that is assuming independence between the two processes. Covariate selection was made using uni- or multivariate Wald tests.

5.2.1. Longitudinal sub-model specification

The biphasic shape of log-PSA was described in a linear mixed model with two functions of time according to previous works [6]: f₁(t) = (1 + t)^α − 1 and f₂(t) = (t)^1+ν/(1 + t)^ν, where α and ν were estimated by profile likelihood (α = −1.2, ν = 0). Thus, these two functions depicted the short term drop in the level of log-PSA after EBRT and the long term linear increase of log-PSA, respectively. By denoting Y_ij = log(PSA_i (t_ij) + 0.1) the log-measure of PSA for the individual i at time t_ij –the natural logarithm transformation is performed to obtain a Gaussian shape for the longitudinal response– the linear mixed sub-model took the form:

$$\begin{array}{c}{Y}_{\mathit{ij}}=Y_i^{\ast }\left(t_{\mathit{ij}}\right)+{\epsilon }_{ij}\\ =({\beta }_0+X_i^{L0\mathrm{\top }}{\beta }_{0,\text{cov}}+b_{i0})+\\ ({\beta }_1+X_i^{L1\mathrm{\top }}{\beta }_{1,\text{cov}}+b_{i1})\times f_1\left(t_{ij}\right)+\\ ({\beta }_2+X_i^{L2\mathrm{\top }}{\beta }_{2,\text{cov}}+b_{i2})\times f_2\left(t_{\mathit{ij}}\right)+{\epsilon }_{\mathit{ij}},\end{array}$$

with $b_i={\left(b_{i0},b_{i1},b_{i2}\right)}^{\mathrm{\top }}~\mathcal{N}\left(0,D\right),D$ unstructured, and ${\epsilon }_i={\left({\epsilon }_{i1},\dots ,{\epsilon }_{i{n}_i}\right)}^{\mathrm{\top }}~\mathcal{N}\left(0,{\sigma }^2{I}_{n_i}\right)$. The covariates $X_i^{L0}$, $X_i^{L1}$ and $X_i^{L2}$ were sub-vectors of the baseline prognostic factors obtained using a backward stepwise procedure. For the sake of brevity, we will speak about PSA dynamics and biomarker’s current level/slope when referring actually to the dynamics of log(PSA + 0.1) and the current level/slope of $Y_i^{\ast }\left(t\right)$, respectively.

5.2.2. Multi-state sub-model specification

In the multi-state sub-part, the determination of prognostic factors and proportionality between baseline intensities was also made by considering no link between the two processes (η = 0) and unspecified baseline intensities (i.e. using a standard semi-parametric multi-state model). The full sub-model considered transition-specific baseline intensities and transition-specific effects of baseline prognostic factors. To reduce the excessive number of parameters to be estimated, proportional baseline intensities were first assumed for some transitions. Clinically, it made sense to consider proportional baseline intensities for transitions leading to local recurrence or hormonal therapy: λ_01,0(t) = exp(−ζ₀₂)λ_02,0(t) = exp(−ζ₁₂)λ_12,0(t); and for the transitions leading to distant recurrence: λ_03,0(t) = exp(−ζ₁₃)λ_13,0(t) = exp(−ζ₂₃)λ_23,0(t). These assumptions were confirmed by the data. We could not make the same assumption for all transitions leading to death because the proportional hazards assumption was not verified. Instead, we chose λ_14,0(t) = exp(−ζ₂₄)λ_24,0(t) and λ_04,0(t) was stratified on the cohort. This procedure reduced the number of baseline intensities to six. A second step consisted in selecting the prognostic factors. Factors with an associated p-value > 0.5 were removed, and common covariate effects on several transitions were considered using multivariate Wald tests. For example, the baseline T-stage category had the same effect on transition intensities 0 → 1, 0 → 3 and 2 → 3. Finally, prognostic factors and the log-coefficients of proportionality between baseline intensities with p-value < 0.1 were selected by using a backward stepwise procedure.

5.2.3. Joint multi-state model specification

In the joint model, log baseline intensities approximated by linear combinations of cubic B-splines with three internal knots replaced the unspecified ones. Note that the first knot was placed at 1 year to take into account the null risk of recurrence before 1 year in these data. The dependence function W_hk,i(b_i, t) was the same for all the transitions h → k and was determined using Wald tests. It resulted that the combination of the true current level and the true current slope of the biomarker fitted at best the relationship between PSA dynamics and the instantaneous risk to transit between health states. Thus, the multi-state sub-model was:

$${\lambda }_{\mathit{hk}}^i(t|b_i)={\lambda }_{hk,0}(t)\text{exp}\mathbf{\{}{X}_{\mathrm{h}k,i}^{S\mathrm{\top }}{\gamma }_{hk}+{\mathbf{\left(}\begin{array}{c}{Y}_i^{\ast }\left(t\right)\\ \partial Y_i^{\ast }\left(t\right)/\partial t\end{array}\mathbf{\right)}}^{\mathrm{\top }}\mathbf{\left(}\begin{array}{c}{\eta }_{hk,\text{lavel}}\\ {\eta }_{hk,\text{slope}}\end{array}\mathbf{\right)}\mathbf{\}},$$

The relations between λ_hk,0(t) and the final $X_{hk,i}^S$, for h, k ∈ {0, …, 4} are indicated in Section 5.2.2 and in Table 3. Note that the covariates that were removed of the joint model specification are not in Table 3.

Table 3.

Parameter estimates, standard errors and p-values in the joint multi-state model on the pooled data (N = 1474).

	Longitudinal Process				Multi-state Process
	Value	StdErr	p-value		Value	StdErr	p-value
β0	−0.26	0.06	< 0.001	γ02,iPSA	0.35	0.08	< 0.001
β0,iPSA	0.80	0.03	< 0.001	γ04,iPSA	0.25	0.08	0.001
β0,cohort	−0.01	0.02	0.541	γ(13,14,23,24,34),iPSA	−0.25	0.08	0.001
β1	0.70	0.14	< 0.001	γ(01,03,23),tstage2	0.92	0.18	< 0.001
β1,iPSA	0.89	0.06	< 0.001	γ(01,03,23),tstage3–4	0.76	0.23	0.001
β1,tstage2	0.38	0.08	< 0.001	γ(12,14,34),tstage2	−0.11	0.25	0.659
β1,tstage3–4	0.47	0.13	< 0.001	γ(12,14,34),tstage3–4	0.33	0.30	0.271
β1,cohort	−0.04	0.04	0.346	γ(03,23)gleason7	0.95	0.25	< 0.001
β2	−0.19	0.04	< 0.001	γ(03,23)gleason8–10	0.07	0.43	0.873
β2,iPSA	0.19	0.02	< 0.001	γ(01,14,24,34),cohort	−0.42	0.06	< 0.001
β2,tstage2	0.14	0.02	< 0.001	γ(13,23),cohort	0.88	0.17	< 0.001
β2,tstage3–4	0.26	0.04	< 0.001	ζ(12,13)	4.19	0.38	< 0.001
β2,gleason7	0.07	0.02	< 0.001	ζ23	3.08	0.53	< 0.001
β2,gleason8–10	0.22	0.04	< 0.001	η01,level	0.36	0.09	< 0.001
β2,cohort	−0.06	0.01	< 0.001	η02,level	0.50	0.07	< 0.001
log(σ)	−1.30	0.01		η03,level	0.42	0.12	< 0.001
				η04,level	−0.15	0.05	0.005
D11	0.37	0.02		η12,level	−0.17	0.10	0.095
D12	0.01	0.01		η13,level	−0.43	0.20	0.033
D13	0.35	0.03		η14,level	0.10	0.14	0.456
D22	0.14	0.01		η23,level	−0.17	0.10	0.081
D23	0.25	0.02		η24,level	0.05	0.05	0.346
D33	1.70	0.09		η34,level	0.02	0.08	0.813
				η01,slope	2.63	0.31	< 0.001
				η02,slope	3.11	0.25	< 0.001
				η03,slope	2.68	0.55	< 0.001
				η04,slope	0.92	0.34	0.007
				η12,slope	2.16	0.63	0.001
				η13,slope	3.44	0.83	< 0.001
				η14,slope	−0.22	1.27	0.864
				η23,slope	1.13	0.68	0.099
				η24,slope	0.21	0.52	0.692
				η34,slope	−0.56	0.78	0.472

D_ij denotes the i j-element of the random effect covariance matrix. γ_{(hk,h′k′),X} denotes the common effect of covariate X on the intensities of transitions h → k and h′ → k′, i.e.γ_{(hk,h′k′),X} = γ_hk,X = γ_h′k′,X. Similarly, ζ_(12,13) = ζ₁₂ = ζ₁₃.

5.3. Results

The parameter estimates of the joint multi-state model are presented in Table 3. These parameters were those selected according to the procedure described previously. The inference was performed using 9 and 9 quadrature points with the two-step Gauss-Hermite quadrature rule. The parameters of the baseline intensities are not shown here for clarity.

The estimated regression parameters in the longitudinal sub-part confirmed that pre-treatment PSA level was associated with the initial PSA level and the biphasic PSA trajectory, T-stage value was associated both with the short term and the long term dynamics while Gleason score was only associated with the long term trajectory. Higher values of these covariates measured at baseline corresponded to higher long term PSA levels. The cohort effect indicated a significant difference between the two cohorts only for the long term PSA evolution, with a greater long term increase of PSA in Vancouver.

For the multi-state process, the model showed that an advanced initial stage was not always associated with the intensities of transitions between health states after adjustment for the PSA dynamics. In particular, the Gleason score had significant effects on two transition intensities only. Moreover, having a high PSA value at baseline was significantly associated with a higher instantaneous risk to directly experience hormonal therapy initiation or death after EBRT, but reduced the intensities of transitions leading to distant recurrence or death after a previous event. A poor (i.e. higher) T-stage category at baseline had globally a deleterious effect on the clinical endpoints. For the transitions from end of EBRT or hormonal therapy initiation to distant recurrence, a patient with a Gleason score of 7 at baseline had a 2.60 = exp(0.954) (95% CI = 1.60–4.20) higher hazard to transit than a patient with a Gleason score < 7. The cohort was significantly associated with the intensities of transitions leading to death after clinical recurrence or hormonal therapy initiation –and the direct transition leading to local recurrence after end of BRT. The instantaneous risk to experience these transitions was higher in BCCA. The cohort effect was also significant, with higher intensities in RTOG 9406, for the direct transitions from local recurrence or hormonal therapy initiation to distant recurrence.

Regarding the association parameters between PSA dynamics (current level and current slope) and clinical progressions, remind that PSA data were collected until the occurrence of the first event. This has an impact on the interpretation of these association parameters. Indeed, because of the focus on the biomarker trajectory before the first event, posterior marker values were extrapolated according to this basal PSA trajectory. We found highly significant deleterious effects of the PSA dynamics on the intensities of transitions from the initial state to all the types of progression (local recurrence, hormonal therapy or distant recurrence). For example, after adjustment for covariates and for the true slope of the biomarker, an increase of one unit of the true biomarker’s level (log PSA without error measurement) induced a 1.43 = exp(0.358) (95% CI = 1.45–1.89) higher risk to experience a local recurrence. These results were expected: in patients with localized prostate cancer and treated by radiotherapy, a persistently high PSA level or/and a strong increase of PSA leads to higher hazard to experience a clinical recurrence or an additional therapy. In contrast, for the direct transition leading to death after radiotherapy, we found a deleterious effect of the current slope and a protective effect of the current level of the biomarker: at a given moment in the initial state, for two patients with the same baseline characteristics and the same slope of log PSA, the one with higher PSA value will be less likely to directly die. In this studied population, an important cause of direct death is induced by comorbidities, because most of death from prostate cancer experienced a documented disease progression before. From the local recurrence, there was large deleterious effect of the current slope of the biomarker for the intensities of transitions leading to the hormonal therapy or the distant recurrence, and there was a borderline significant protective effect of the current level for the intensity of transition leading to the distant recurrence. From the hormonal therapy or the distant recurrence, there was no significant effect of the PSA dynamics on the hazard to change state. This was also clinically sensible, as it reflects that progression in these advanced stages is not linked anymore to PSA increase. In practice, criteria other than PSA are considered in this phase of the disease, such as the PCWG2 criteria [26]. Moreover, deaths in patients with hormonal therapy might be explained by cardiac toxicity due to HT.

5.4. Diagnostics

The parameter estimates of the joint multi-state model were validated by several graphical tools presented in Figure 4. For the longitudinal sub-model, the plotted standardized conditional residuals versus fitted values of the biomarker confirmed the homoscedasticity of the conditional errors. Subject-specific predictions were also compared to observations by plotting the average values by time intervals based on the deciles of the observation times. 95% confidence intervals of the observed values were added and confirmed the very good fit of the model to the longitudinal data. For the multi-state sub-model, we focused on P(0, t) = {P_hk(0, t)}, the matrix of transition probabilities between times 0 and t. We compared our parametric estimator (obtained with the average of the predicted individual transition probabilities from the joint multi-state model) to the Aalen-Johansen estimator (non-parametric estimator of the transition probabilities), both using product integrals. This comparison is fully discussed and detailed in Web Appendix D. These comparisons showed the overall good performances of the joint multi-state model in terms of fit for the transition probabilities, with the exception for transition 1 → 2 for which the immediate pike after EBRT could not be correctly captured by splines.

Figure 4.

Goodness-of-fit plots for the longitudinal process (a,b) and the multi-state process (c).

6. Discussion

The joint model for the longitudinal biomarker PSA and multi-state clinical progression data provides a complete model of prostate cancer progression which takes into account both classical prognostic factors and PSA dynamics, in order to study factors that influence the transition intensities between clinical health states. The implementation is easy as it relies on mstate and JM packages. The multi-state data are prepared with mstate package, and a slightly modified jointModel() function carries out the estimation procedure. The estimation program has been validated by simulations, with very good performances. Even when the number of subjects experiencing some direct transitions was low, coverage rates remained satisfactory. The simulations underlined however some bias in the estimates when the dimension of the random effects increased (≥ 3 random effects) and/or too few quadrature points (3 points in particular) were used. To address this problem, we proposed a two-step procedure which updates the location of the quadrature points and improves the quality of the approximation of the integral over the random effects while keeping a small number of quadrature points. This also reduces substantially the computation time compared to a pseudo-adaptive Gauss-Hermite rule with a much larger number of quadrature points. With this new rule, models with 3 random effects can be correctly estimated using 9 and 9 quadrature points. Diagnostic graphical tools were also proposed to assess the goodness-of-fit of the model (methodology detailed in Web Supplementary Material D).

The application confirmed that the PSA dynamics strongly impacted the instantaneous risk to experience a clinical recurrence or hormonal therapy initiation after the end of radiotherapy. The current slope of the biomarker had also a highly significant deleterious effect on the hazard to transit from local recurrence to hormonal therapy or distant recurrence. Conversely, extrapolating the biomarker’s dynamics did not impact anymore the transition intensities from the hormonal therapy initiation state or the distant recurrence state. This highlights that in the advanced cancers, the PSA –and especially the collected measures prior to the first event– is not of importance anymore. In these situations, other criteria have to be monitored. Note that data posterior to the first clinical recurrence or the hormonal therapy were not available in our application. When available, it would be of great interest to include them in order to capture the effect of the actual marker dynamics rather than the basal trajectory. However it would also usually imply a much more complex model for the longitudinal marker as the dynamics might change.

Previous works in prostate cancer had found a strong association between slope of log-PSA and any clinical recurrence (see Sène et al. [27]; Taylor et al. [28]), by considering all the recurrences in a composite event and the hormonal therapy as a time-dependent covariate. The limit of these approaches was that in practice considering the type of progression is of major importance as the care greatly depends on the type of risk the patient has. The joint multi-state model formalizes this need. In the same way as it was done with a single event (see Proust- Lima and Taylor [29]; Rizopoulos [30]), individualized dynamic predictions of each type of progression could be derived from this model in order to precisely quantify the risk of each type of progression according to the PSA history. For example, the cumulative probability for subject i to reach state k between times s and t, s ≤ t, given he was in state h at time s, could be expressed as: ${\pi }_{hk}^i\left(s,t\right)={\displaystyle {\int }_s^t\text{Pr}\left(E_i\left(u\right)=k|E_i\left(s\right)=h,Y_i^{\left(s\right)},X_i^{L\left(s\right)},X_i^S\right)}$ du, with $Y_i^{\left(s\right)}$ the history (i.e. collected measures) of the marker up to time s, $X_i^{L\left(s\right)}$ the history of the longitudinal sub-model covariates until time s, and $X_i^S=\left\{X_{hk,i}^S\right\}$ the matrix containing the prognostic factors for all the state transitions.

In this article, we made several assumptions. First, we assumed a continuous and Markov multi-state process as it was clinically relevant for the progression of prostate cancer after treatment. However, in other contexts, a semi-Markov process which considers the time spent in the current state could be defined as well. In dementia for example, the multi-state process might include three states (healthy, demented and dead), and consider the time spent in the demented state before death (see Commenges et al. [31]). The joint multi-state model we proposed and its associated implemented function handle for semi-Markov. Second, through the (semi-)Markov assumption, we assume that the dependency between the transition times for a given subject is entirely explained by the prognostic factors and the marker dynamics. This assumption could be relaxed by including some frailty term in the multi-state model. However, Putter and van Houwelingen [32] pointed out that identifiability of multi-state models with frailties is weak and the interpretation becomes not obvious. Third, we chose the nature of the dependence function using goodness-of-fit measures in the application, but other strategies could be used. For example this choice might rely on predictive accuracy measures when focusing on prediction (see Sène et al. [33]). Finally, there was no delayed entry in the prostate cancer application. However the method implicitly handles delayed entry by conditioning the log-likelihood on the state at entry in the study, as it was done by Commenges [34].

In summary, we introduce here a first joint model for longitudinal and multi-state clinical progression data. We showed that this model can easily be implemented under R and can be applied in practice through an example, the prostate cancer progression, which is one of many biomedical areas in which such data are collected. This model that captures the complete information about the progression opens to much more precise knowledge of diseases and specific dynamic predictions.