Bias in progression‐free survival analysis due to intermittent assessment of progression

Abstract

Cancer clinical trials are routinely designed to assess the effect of treatment on disease progression and death, often in terms of a composite endpoint called progression‐free survival. When progression status is known only at periodic assessment times, the progression time is interval censored, and complications arise in the analysis of progression‐free survival. Despite the advances in methods for dealing with interval‐censored data, naive methods such as right‐endpoint imputation are widely adopted in this setting. We examine the asymptotic and empirical properties of estimators of the marginal progression‐free survival functions and associated treatment effects under this scheme. Specifically, we explore the determinants of the asymptotic bias and point out that there is typically a loss in power of tests for treatment effects. Copyright © 2015 John Wiley & Sons, Ltd.

1. Introduction

A fundamental goal in oncology is the reduction of mortality due to cancer. Therapeutic advances for many cancers and the increasing pressure to evaluate experimental treatments in a timely and cost‐effective manner have made it challenging to design adequately powered trials based on the time from randomization to death. This has led to increased use of the composite progression‐free survival endpoint 1, where the nominal response is the time from randomization to the first of progression or death. From 2005 to 2009 inclusive, for example, just over one‐quarter of the trials published on breast, colorectal and non‐small‐cell lung cancer in the Journal of Clinical Oncology assess treatment effects based on the composite endpoint of progression‐free survival 2.

The utility and interpretation of findings based on progression‐free survival are currently being discussed in the oncology literature 3. Miksad et al. 4 point out that because interest primarily lies in the effect of treatment on overall survival, there is an implicit assumption that progression‐free survival is a surrogate for overall survival. These authors conduct a scientific review to examine the empirical evidence of an association between findings from progression‐free survival and overall survival analyses; they conclude that while there is an association between the findings, the predictive accuracy is not high. Sidhu et al. 5 study this issue in metastatic colorectal cancer trials involving fluoropyrimidine‐based regimens and found a higher correlation, leading them to conclude that progression‐free survival is a valid surrogate for overall survival in this context. In another context, Ballman et al. 6 examine the association between 6‐month progression‐free survival and overall survival at 12 months. With a view to gaining insight into the structural nature of the relation between these two endpoints, Broglio and Berry 7 decompose the effect of treatment on overall survival into components, one of which relates to progression‐free survival. Saad et al. 8 emphasize the importance of a clear definition of progression when analysed on its own or as part of a composite endpoint.

Frydman and Szarek 9 point out that the classical illness‐death model (Figure 1) offers a natural framework for joint consideration of a non‐terminal event, such as progression, and the terminal event death. In the current setting, we consider individuals to be in state 0 at the time of randomization when they are progression free and alive. Individuals enter state 1 upon progression and subsequently enter state 2 upon death. The possibility of death without progression is accommodated by direct transition from state 0 to state 2. Let T _jk denote the potential j→k transition time. If T ₀₁<T ₀₂, we let T ₁=T ₀₁ denote the time of entry to state 1. The time of entry to state 2 is the overall survival time denoted by T ₂=I(T ₀₂<T ₀₁)T ₀₂+I(T ₀₁≤T ₀₂)T ₁₂. The progression‐free survival time is the time of exiting state 0 denoted by T= min(T ₀₁,T ₀₂)= min(T ₁,T ₂).

Figure 1.

A three‐state model for joint consideration of progression and death.

Challenges arise when progression‐free survival is used as an endpoint, as progression status is only observed at periodic assessment times and the time of progression is at best interval censored. The time to the composite progression‐free survival endpoint is then subject to a hybrid censoring scheme involving interval censoring for progression and right censoring for death.

Figure 2(a) shows a timeline diagram where T ₁ and T ₂ are the times of progression and death, respectively; A _k,k = 1,2,… denote the assessment times; and C is a right censoring time. In this scenario, progression is first detected at A ₂, so one knows that T lies in the interval (A ₁,A ₂]. The most common strategy for dealing with this type of data, however, is to use right‐endpoint imputation whereby the surrogate S = A ₂ is used in lieu of the event time T and standard survival analysis methods are adopted; this approach is recommended by the FDA 10. A further complication arises when death occurs without prior evidence of progression (Figure 2(b)) as it is unknown whether or not progression occurred between the last negative assessment (A ₂) and the time of death. The convention in this case is to assume individuals have not progressed and hence use S = T ₂ as the progression‐free survival time (e.g. see Dejardin et al. 11). The strong implicit assumptions associated with the approaches described earlier warrant careful consideration when dealing with the intermittently observed progression component in the composite progression‐free survival endpoint.

Figure 2.

Timeline diagrams for progression (T ₁), death (T ₂), assessments (A ₁ and A ₂) and censoring (C).

Given the vast number of cancer trials using progression‐free survival as the primary endpoint, the challenges in interpreting findings based on composite endpoints and the seemingly self‐evident shortcomings of using the documented time of progression rather than the actual progression time, it is surprising that this problem has not received more attention. Frydman and Szarek 9 and Lefonndré et al. 12 highlight and examine the biases that arise from some form of imputation in this setting. Panageas et al. 13 discuss the role of the assessment schedule on inferences regarding progression‐free survival analysis and discuss, in particular, the phenomenon that the Kaplan–Meier (KM) estimate tends to drop around the time of scheduled assessments in a way that reflects both the risk of progression and the assessment process. Binder and Schumacher 14 consider modelling the time to progression and examine the impact of censoring progression times at death for individuals not observed to have progressed. Lefonndré et al. 12 advocate suitable methods for interval‐censored data when modelling the joint distribution of progression and death times. Much of this work has been based on simulation studies.

In this article, we examine the asymptotic and empirical properties of estimators arising from use of the surrogate ‘observed’ progression‐free survival times as is customarily carried out under the intermittent assessment scheme. We consider the settings where the assessment times are pre‐scheduled, but there is modest variation between individuals in precisely when they occur, to reflect data arising in clinical trials. We also consider the setting in which assessment times are governed by an independent and possibly progression‐dependent stochastic process to correspond to the less regulated assessment schemes of observational cohort studies. We formulate models for each component of the composite endpoint (progression and death) in the framework of an illness‐death model with proportional intensities so that the proportional hazards assumption is satisfied for the endpoint of progression‐free survival. Asymptotic calculations are carried out by considering an enlarged state space that allows joint consideration of the disease process and the assessment process. From this stochastic model, the distribution of the ‘observed’ progression‐free survival time is derived, where it is defined by right‐endpoint imputation when progression is detected and the survival time otherwise. We contrast this distribution with that of the latent progression‐free survival time in a variety of scenarios. We also use the theory of misspecified Cox models to derive the asymptotic biases of the estimated treatment effects from a Cox proportional hazard model for progression‐free survival 15, 16, 17.

The remainder of this paper is organized as follows. In Section 2, we introduce further notation for the three‐state illness‐death model, and we discuss issues in the use of ‘observed’ progression‐free survival when progression is intermittently observed in Section 3. We derive the expected partial score functions, which are central to the derivation of the asymptotic bias in estimates of treatment effect from the Cox proportional hazards model. We also derive the asymptotic power associated with Wald‐type tests of the null hypothesis of no treatment effect and demonstrate the potential reduction in power from adopting right‐endpoint imputation. Empirical investigations are conducted in Section 4 to illustrate the finite sample properties, and concluding remarks are given in Section 5.

2. The Markov illness‐death model for progression and death

We let Z(t) denote the state occupied at time t, and {Z(u),0 < u} represent the corresponding stochastic process. At time t > 0, the history of the process, denoted as H(t)={Z(u),0 < u < t}, contains information on the timing and nature of any transitions over (0,t). Transitions among the states $S=\{0,1,2\}$ at any time t are governed by the transition intensities

$$\underset{\mathrm{\Delta t}\to 0}{\lim }\frac{\mathrm{P}\left(Z\right(t+\mathrm{\Delta t})=k|H\left(t\right))}{\mathrm{\Delta t}}=Y_j\left(t\right){\lambda }_{\mathrm{jk}}\left(t\right|H\left(t\right)),j<k\in S,$$ (1)

where Y _j(t) = I(Z(t ⁻) = j) indicates that the individual is ‘at risk’ of a transition out of state j at time t, j = 0,1.

If the transition intensities depend on the process history only through state occupied at t ⁻, we write

$${\lambda }_{\mathrm{jk}}\left(t\right|H\left(t\right))={\lambda }_{\mathrm{jk}}(t),$$

and the process is Markov. We can then let $\mathbb{A}\left(t\right)=\left[{\lambda }_{\mathrm{jk}}\left(t\right)\right]$ denote the transition intensity matrix, where ${\mathbb{A}}_{\mathrm{jk}}\left(t\right)={\lambda }_{\mathrm{jk}}\left(t\right)$ for $j<k\in S,{\mathbb{A}}_{\mathrm{jj}}\left(t\right)=-\sum _{k>j}{\lambda }_{\mathrm{jk}}\left(t\right)$, and other elements are zero. Let $\mathbb{P}(s,t)=\left[p_{\mathrm{jk}}\right(s,t\left)\right]$ denote the transition probability matrix with elements p _jk(s,t) = P(Z(t) = k|Z(s) = j),s < t. The matrix $\mathbb{P}(s,t)$ satisfies the Kolmogorov forward differential equation

$$\frac{\mathrm{d}}{\mathrm{d}t}\mathbb{P}(s,t)=\mathbb{P}(s,t)\mathbb{A}\left(t\right),t>s,$$ (2)

which enables one to express the transition probabilities in terms of intensity functions 18. When the process is time homogeneous, the calculation becomes much simpler and $\mathbb{P}\left(t\right)=\exp \left(\mathbb{At}\right)$. For the three‐state illness‐death models shown in Figure 1, the explicit expressions for p ₀₀(s,t) and p ₀₁(s,t) are

$$\begin{array}{cc}{p}_{00}(s,t)& =\exp \left(-\underset{s}{\overset{t}{\int }}\left[{\lambda }_{01}\right(u)+{\lambda }_{02}(u\left)\right]\mathrm{d}u\right),\\ p_{01}(s,t)& =\underset{s}{\overset{t}{\int }}{\lambda }_{01}\left(u\right)\exp \left(-\underset{s}{\overset{u}{\int }}\left[{\lambda }_{01}\right(v)+{\lambda }_{02}(v\left)\right]\mathrm{d}v\right)\exp \left(-\underset{u}{\overset{t}{\int }}{\lambda }_{12}\left(v\right)\mathrm{d}v\right)\mathrm{d}u.\end{array}$$ (3)

Note that the survival function for the progression‐free survival endpoint is then

$$\mathcal{F}\left(t\right)=P(T>t)=p_{00}(0,t),t>0,$$ (4)

with respective hazard function $h\left(t\right)=-\mathrm{d}\log \left(\mathcal{F}\right(t\left)\right)/\mathrm{d}t={\lambda }_{01}\left(t\right)+{\lambda }_{02}\left(t\right)$, the sum of the cause‐specific hazard functions for 0→1 and 0→2 transitions.

At times in what follows, it will also be useful to consider the counting process formulation of multistate processes, and with that in mind, we define N _jk(t) as the number of j→k transitions over (0,t] and let Δ N _jk(t) = N _jk(t + Δ t ⁻) − N _jk(t ⁻) count the number of j→k transitions over the interval [t,t + Δ t). Then $\mathrm{d}{N}_{\mathrm{jk}}\left(t\right)=\underset{\mathrm{\Delta t}\to 0}{\lim }\Delta N_{\mathrm{jk}}\left(t\right)$ indicates the occurrence of a j→k transition at time t.

3. Progression‐free survival with intermittent assessment schemes

3.1. Distribution of imputed progression‐free survival time

When interest lies in evaluating treatment effects on the basis of progression‐free survival, standard survival models are often fitted with the time T= min(T ₁,T ₂) in mind. As discussed in Section 2, progression status is only determined at periodic assessment times, and hence, among individuals for whom progression is detected, the progression time is interval censored. For such individuals, the convention is to use the assessment time at which progression is detected as the progression‐free survival time. For individuals dying without progression having previously been detected, it is unknown whether or not progression occurred, and in this case, it is conventional to assume they were progression‐free at the time of death and to take the time of death as the progression‐free survival time. These conventions lead to invalid inference because the distributions of the adopted event time and the actual progression‐free survival time differ. In what follows, we formally construct a probability model that represents the process generating the available data. We then use this model to evaluate the performance of methods used in this situation.

Let C _A denote an administrative censoring time and R _i a random right censoring time for individual i, so that the survival status is known over the interval (0,C _i], where C _i= min(C _A,R _i). Suppose progression status is assessed at a sequence of times 0 = a _i0<a _i1<⋯<a _{iK i}<C _i. The surrogate for the progression‐free survival time T is given by

$$S=\min \{a_{\mathrm{ik}};I\left(Z\left(a_{\mathrm{ik}}\right)=1\right),k=1,\dots ,K_i\}·I\left(Z\left(a_{i{K}_i}\right)=1\right)+T_2·I\left(Z\left(a_{i{K}_i}\right)=0\right),$$ (5)

which takes the value of the assessment time if progression is detected, and the time of death otherwise. The progression‐free survival time is considered right censored at the last negative assessment if the final assessment is negative and death has not been observed during the course of follow‐up 11. We derive the distribution function for the surrogate event time S under two different intermittent assessment schemes, deferring discussion of covariates until Section 3.2.

The first scenario is designed to mimic the setting of a randomized trial and the second a less regular assessment scheme such as one might see in a cohort study. The prostate cancer trial by Scher et al. 19 is a suitable example in which progression is evaluated every month during the first 6 months of follow‐up and every 3  months thereafter. For ease of exposition, we assume there is an assessment at the time of randomization denoted a ₀=0, and K regularly spaced follow‐up assessments are scheduled over (0,C _A] at a _k=k C _A/K,k = 1,…,K. According to (5), the surrogate progression‐free survival time S is one of these fixed assessment times if progression is detected, but it otherwise takes on any value over (0,C _A] if death is observed. This S is a mixed‐type non‐negative random variable with positive mass at a finite number of discrete points (at the assessment times) and a cumulative distribution function

$$F_S\left(s\right)=P(S\le s)=\underset{0}{\overset{s}{\int }}f\left(u\right)\mathrm{d}u+\sum _{k:a_k\le s}P(S=a_k),$$

where

$$\begin{array}{cc}f\left(u\right)& =p_{00}(1a_0,a_{k-1})\left[\sum _{j=0}^1{p}_{0j}(a_{k-1},u^{-}){\lambda }_{j2}\left(u\right)\right]\text{for}u\in (a_{k-1},a_k),\\ P(S=a_k)& =p_{00}(a_0,a_{k-1})p_{01}(a_{k-1},a_k),\end{array}$$

and p ₀₀(u,v) and p ₀₁(u,v) are given in (3). The cumulative function is a complicated time‐varying function of the baseline intensities in the three‐state multiplicative illness‐death model characterizing the true underlying process of progression and death, as is the hazard function h _S(s).

For cohort studies, point process models offer a useful framework for characterizing the stochastic assessment process. We let A _k denote the random time of the kth assessment, k = 1,2,…, and $N_a\left(t\right)=\sum _{k=1}^{\infty }I(A_k\le t)$ count the number of follow‐up assessments occurring over (0,t]. If Y(s) = I(s≤C) and the history of the joint process is defined as H ^†(t)={(Z(a _k),a _k),k = 0,…,N _a(t ⁻);I(s≤T ₂),Y(s),0 < s < t} for an arbitrary individual, then their visit intensity is defined as

$$\underset{\mathrm{\Delta t}\to 0}{\lim }\frac{\mathrm{P}\left(\Delta N_a\right(t+\mathrm{\Delta t})=1|H^{†}\left(t\right))}{\mathrm{\Delta t}}=Y\left(t\right){\lambda }^{†}\left(t\right|H^{†}\left(t\right)).$$ (6)

A joint model for the response, assessment and censoring processes can be constructed by first expanding the state space. Specifically, we define a new multistate process {Z ^†(s),0 < s} with a state space $S^{†}=\{V_0,V_1,\dots ,V_1^p,V_2^p,\dots ,P_0,P_1,\dots ,D_0,D_1,\dots ,\dots \}$ as depicted in Figure 3. The subscript on the states V _k,k = 1,2,…, reflects the number of assessment visits for which progression was not detected; for the states $V_k^p,k=1,\dots$, the superscript p designates an assessment made post‐progression. Thus, an individual is in state V _k at time t if they are alive and progression‐free and N _a(t) = k (i.e. A _k≤t< min(T ₁,T ₂,A _{k + 1})). They are in state P _k at time t if they progressed after the kth assessment but have not yet experienced the (k + 1)st assessment or died (i.e. A _k<T ₁≤t< min(T ₂,A _{k + 1})). An individual is in state D _k at time t if I(A _k<T ₂< min(t,A _{k + 1})) and in state $V_k^p$ at time t if I(A _k<T ₁<A _{k + 1}≤t < T ₂),k = 0,1,2,…. Following the occurrence of the kth assessment, the next event to occur (visit k + 1, progression or death) is governed by a competing risk process; transitions to the right (i.e. V _k→V _{k + 1}) occur with the cause‐specific intensity λ ^†(t|H ^†(t)) = α(t). Downward transitions from the V _k state (i.e. V _k→P _k and V _k→D _k) correspond to transitions out of state 0 in the original three‐state illness‐death process and hence have the same transition intensities λ ₀₁(t) and λ ₀₂(t); likewise, the P _k→D _k transitions have intensity λ ₁₂(t) under a Markov model. Finally, the $P_k\to V_{k+1}^p$ transition corresponds to the occurrence of the (k + 1)st assessment following progression and has intensity λ ^†(t∣T ₁<t,H ^†(t)) = α _p(t). If we assume the assessment process is independent of the disease process, then α _p(t) = α(t), but otherwise, we may assign a different intensity, such as α _p(t) = α(t) exp(ρ), to reflect increase intensity of the assessment process post‐progression if, for example, ρ > 0. The latter may be reasonable if clinic visits are more likely if symptoms associated with progression become evident; note that such a process would violate the sequential missing at random assumption 20 and invalidate even methods based on interval‐censored data 21.

Figure 3.

A multistate diagram for joint consideration of progression, death and recurrent assessment times.

We define $\mathcal{V}$ as the set of states V ₀,V ₁,…, and define ${\mathcal{V}}^p,\mathcal{P}$ and $\mathcal{D}$ similarly. Under such a setup, if K is large enough, the survival function for the surrogate progression‐free survival time S is simply $P\left(Z^{†}\right(s)\notin {\mathcal{V}}^p\cup \mathcal{D})$. In other words, the survival function ${\mathcal{F}}_S\left(s\right)$ can be expressed as a function of the transition probabilities of the multistate process given in Figure 3 such that

$${\mathcal{F}}_S\left(s\right)=1-P\left(Z^{†}\left(s\right)\in {\mathcal{V}}^p\mid Z^{†}\left(0\right)=0\right)-P\left(Z^{†}\left(s\right)\in \mathcal{D}\mid Z^{†}\left(0\right)=0\right).$$ (7)

The transition probabilities necessary to compute this can be obtained by the Kolmogorov differential equation of Section 2 but can also be calculated directly from the probability of particular sample paths. For example, for an individual to be in state $V_k^p$ at time s, k − 1 assessments have to occur prior to progression, and the individual must survive to the kth assessment at A _k<s. Over [0,s], the individual thus follows the path $V_0\to \dots \to V_{k-1}\to P_{k-1}\to V_k^p$ at times 0 = a ₀<⋯<a _{k − 1}<t ₁<a _k≤s, so the probability $P\left(Z^{†}\left(s\right)=V_k^p\mid Z^{†}\left(0\right)=0\right)$ takes form

$$\begin{array}{c}\underset{0}{\overset{s}{\int }}\underset{0}{\overset{a_k}{\int }}\underset{0}{\overset{t_1}{\int }}\cdots \underset{0}{\overset{a_2}{\int }}\left[\prod _{j=1}^{k-1}\alpha \left(a_j\right)\right]{\lambda }_{01}\left(t_1\right){\alpha }_p\left(a_k\right)\exp \left[-\underset{0}{\overset{t_1}{\int }}\left({\lambda }_{01}\left(u\right)+{\lambda }_{02}\left(u\right)+\alpha \left(u\right)\right)\mathrm{d}u\right]\\ \times \exp \left[-\underset{t_1}{\overset{a_k}{\int }}\left({\lambda }_{12}\left(u\right)+{\alpha }_p\left(u\right)\right)\mathrm{d}u\right]\mathrm{d}{a}_1\dots \mathrm{d}{a}_{k-1}\mathrm{d}{t}_1\mathrm{d}{a}_k.\end{array}$$ (8)

An expression for P(Z ^†(s) = D _k∣Z ^†(0) = 0) is derived similarly by noting that over [0,s], an individual must follow either the path V ₀→⋯→V _k→D _k with transition times 0 = a ₀<⋯<a _k<t ₂≤s, or path V ₀→…→V _k→P _k→D _k with transitions at times 0 = a ₀<⋯<a _k<t ₁<t ₂≤s. Hence, P(Z ^†(s) = D _k∣Z ^†(0) = 0) takes the form

$$\begin{array}{c}\underset{0}{\overset{s}{\int }}\underset{0}{\overset{t_2}{\int }}\cdots \underset{0}{\overset{a_2}{\int }}\left[\prod _{j=1}^k\alpha \left(a_j\right)\right]{\lambda }_{02}\left(t_2\right)\exp \left[-\underset{0}{\overset{t_2}{\int }}\left({\lambda }_{01}\left(u\right)+{\lambda }_{02}\left(u\right)+\alpha \left(u\right)\right)\mathrm{d}u\right]\mathrm{d}{a}_1\dots \mathrm{d}{a}_k\mathrm{d}{t}_2\\ +\underset{0}{\overset{s}{\int }}\underset{0}{\overset{t_2}{\int }}\underset{0}{\overset{t_1}{\int }}\cdots \underset{0}{\overset{a_2}{\int }}\left[\prod _{j=1}^k\alpha \left(a_j\right)\right]{\lambda }_{01}\left(t_1\right){\lambda }_{12}\left(t_2\right)\exp \left[-\underset{0}{\overset{t_1}{\int }}\left({\lambda }_{01}\left(u\right)+{\lambda }_{02}\left(u\right)+\alpha \left(u\right)\right)\mathrm{d}u\right]\\ \times \exp \left[-\underset{t_1}{\overset{t_2}{\int }}\left({\lambda }_{12}\left(u\right)+{\alpha }_p\left(u\right)\right)\mathrm{d}u\right]\mathrm{d}{a}_1\dots \mathrm{d}{a}_k\mathrm{d}{t}_1\mathrm{d}{t}_2\end{array}$$ (9)

When the multistate process is time homogeneous where λ _jk(t) = λ _jk and the assessment process is ignorable and time homogeneous (α(t) = α _p(t) = α), expressions (8) and (9) can be further simplified. Substituting (8) and (9) in (7) results in an expression for ${\mathcal{F}}_S\left(s\right)$, and the hazard h _S(s) is no longer a simple sum of λ ₀₁(s) and λ ₀₂(s).

We compare the survival functions for progression‐free survival time (4) and the surrogate progression‐free survival time (7) under various parameter configurations for the setting where the assessments arise from a point process. We assume time‐homogenous transition intensities ${\lambda }_{\mathrm{jk}}\left(t\right)={\lambda }_{\mathrm{jk}},j<k\in S$, with values set such that (i) P(T ₀₁<T ₀₂) = λ ₀₁/(λ ₀₁+λ ₀₂) equals a desired probability of progression, (ii) λ ₁₂/λ ₀₂=1.5 corresponds to an increased risk of death following progression and (iii) π _A=P(T > C _A) = p ₀₀(0,C _A) = 0.20 gives a 20% administrative censoring rate for the progression‐free survival endpoint; we set C _A=1 without loss of generality. The random right censoring time R _i is assumed to follow an exponential distribution with the hazard set to yield the desired net censoring rates (π _N), where π _N=P(C _i<T) with C _i= min(C _A,R _i). We set α(t) = α and α _p(t) = α exp(ρ) with ρ={log1, log2} for an progression independent or dependent assessment process.

Figure 4(a) contains a plot of the survival functions for the surrogate (black dashed line) and actual (black solid line) progression‐free survival times for the case when P(T ₀₁<T ₀₂) = 0.8, and the assessment process is event independent with a time homogeneous rate α = 2. The asymptotic bias in the estimator of the median progression‐free survival time is evident in Figure 4(a) from the distance between the vertical lines; the median progression‐free survival time is overestimated by a factor of (0.31 − 0.12)/0.12 = 2.6. Similarly, a progression‐free survival analysis based on the surrogate time leads to an over‐estimation of the probability of being event free. We also find that when all other factors are fixed, the extent of over‐estimation is greater when assessments are less frequent (e.g. comparing the biases when α = 2 versus when α = 4). Finally, the asymptotic bias decreases as P(T ₀₁<T ₀₂) decreases below 0.8 because as the probability of failure due to death increases, the probability that the actual failure time will be recorded increases.

Figure 4.

Survival functions for the surrogate progression‐free survival time (dotted) and the actual progression‐free survival time (solid) when the assessment process is (a) event independent and (b) event dependent; here, P(T ₀₁<T ₀₂) = 0.8,α(t) = α = 2,λ ₁₂/λ ₀₂=1.5 and π _A=0.2.

Figure 4(b) displays an analogous plot to Figure 4(a) for the case of an event‐dependent assessment process with ρ= log2; here, the intensity for the next assessment doubles after progression occurs. The findings are similar, but the biases are lower because the lag from progression to its detection is stochastically smaller.

3.2. Cox regression based on the surrogate progression‐free survival time

Treatment effects are naturally characterized by multiplicative intensity models of the form

$${\lambda }_{\mathrm{jk}}(t\mid x)={\lambda }_{\mathrm{jk}0}\left(t\right)\exp \left(x^{\prime }{\beta }_{\mathrm{jk}}\right),$$ (10)

where λ _jk0(t) is the baseline rate of j→k transitions and $\beta ={({\beta }_{\mathrm{jk}}^{\prime },j<k\in S)}^{\prime }$ is the vector of regression parameters. For simplicity, we focus on a binary treatment indicator x, so that the hazard ratio for the progression‐free survival time is

$$\frac{h(t\mid x=1)}{h(t\mid x=0)}=\frac{{\lambda }_{010}\left(t\right)\exp \left({\beta }_{01}\right)+{\lambda }_{020}\left(t\right)\exp \left({\beta }_{02}\right)}{{\lambda }_{010}\left(t\right)+{\lambda }_{020}\left(t\right)}.$$ (11)

The proportionality assumption holds for h(t) under either (i) β ₀₁=β ₀₂ or (ii) λ ₀₁₀(t) = λ ₀₂₀(t) 22, 23. Under the first scenario, the treatment effect on progression‐free survival is the same as the treatment effect on progression and death, β = β ₀₁=β ₀₂. The second scenario leads to a treatment effect on progression‐free survival given by β= log[(exp(β ₀₁)+ exp(β ₀₂))/2]. When neither of these two conditions are satisfied, the hazard ratio (11) will be a complicated time‐varying function of the baseline intensities and the treatment effects for progression and death.

We consider the setting when proportionality holds for the progression‐free survival endpoint under condition (i) and a semi‐parametric regression model for the hazard takes a form h(t;x) = h ₀(t) exp(x β), where h ₀(t) is the baseline hazard, x is a binary treatment indicator and β is the true treatment effect. While this is no doubt unrealistic, we do so because our primary objective is to evaluate the impact of using the surrogate event time S in a Cox regression model under the intermittent assessment scheme. If neither condition (i) nor (ii) is satisfied, the Cox model for T is misspecified to start with, and it will be difficult to determine the role of the inspection process in the results.

To distinguish the true model from the one for the surrogate, we write

$$h_S(s;x)=h_{S0}\left(s\right)\exp \left(\mathrm{x\gamma }\right),s>0,$$ (12)

for the working Cox model based on the surrogate time S with baseline hazard h _S0(s) and treatment effect γ. Let $Y_i^{†}\left(s\right)=I(s\le S_i)$ be the at‐risk indicator and dN _i(s) = I(S _i=t) be the event indicator, which are all defined based on the imputed event time S. If Y _i(s) = I(s≤C _i) indicates individual i is still under observation, then ${\overline{Y}}_i\left(s\right)=Y_i\left(s\right)Y_i^{†}\left(s\right)$ indicates they are under observation and at risk of the composite event. The partial likelihood 24, 25 leads to a score function for γ of the form $U\left(\gamma \right)=\sum _{i=1}^n{U}_i\left(\gamma \right)$, where

$$U_i\left(\gamma \right)=\underset{0}{\overset{\infty }{\int }}{\overline{Y}}_i\left(s\right)\left[X_i-\frac{R^{\left(1\right)}(\gamma ;s)}{R^{\left(0\right)}(\gamma ;s)}\right]\mathrm{d}{N}_i\left(s\right),$$ (13)

$R^{\left(1\right)}(\gamma ,s)=\sum _{i=1}^n{\overline{Y}}_i\left(s\right)X_i\exp \left(X_i\gamma \right)$ and $R^{\left(0\right)}(\gamma ,s)=\sum _{i=1}^n{\overline{Y}}_i\left(s\right)\exp \left(X_i\gamma \right)$. If $\hat{\gamma }$ denotes the estimate of γ by solving U(γ) = 0, then $\hat{\gamma }\to {\gamma }^{*}$ in probability, where γ ^* is the solution to

$$\underset{0}{\overset{\infty }{\int }}\left\{E\left[{\overline{Y}}_i\right(s\left)X_i\mathrm{d}{N}_i\right(s\left)\right]-\frac{r^{\left(1\right)}(\gamma ,s)}{r^{\left(0\right)}(\gamma ,s)}E\left[{\overline{Y}}_i\right(s\left)\mathrm{d}{N}_i\right(s\left)\right]\right\}=0,$$ (14)

with r ⁽⁰⁾(γ,s) = E{R ⁽⁰⁾(γ,s)} and r ⁽¹⁾(γ,s) = E{R ⁽¹⁾(γ,s)}, respectively 16. Lin and Wei 17 prove that $\sqrt{n}(\widehat{\gamma }-{\gamma }^{*})$ is asymptotically normal with

$$\mathrm{var}\left(\sqrt{n}(\widehat{\gamma }-{\gamma }^{*})\right)=\left[A^{-1}\right({\gamma }^{*}\left)\right]\left[B\right({\gamma }^{*}\left)\right]\left[A^{-1}\right({\gamma }^{*}\left)\right]\prime ,$$ (15)

where

$$\begin{array}{cc}A\left(\gamma \right)& =\underset{0}{\overset{\infty }{\int }}E\left\{{\overline{Y}}_i\left(s\right)\left[\frac{r^{\left(1\right)}(\gamma ,s)}{r^{\left(0\right)}(\gamma ,s)}-{\left(\frac{r^{\left(1\right)}(\gamma ,s)}{r^{\left(0\right)}(\gamma ,s)}\right)}^2\right]\mathrm{d}{N}_i\left(s\right)\right\},\\ B\left(\gamma \right)& =\underset{0}{\overset{\infty }{\int }}E\left\{{\overline{Y}}_i\left(s\right){\left[X_i-\frac{r^{\left(1\right)}(\gamma ,s)}{r^{\left(0\right)}(\gamma ,s)}\right]}^2\mathrm{d}{N}_i\left(s\right)\right\}.\end{array}$$

We define γ ^*−β as the asymptotic bias.

The expectations in (14) and (15) are taken with respect to the distribution of the surrogate time S with expressions $E\left\{{\overline{Y}}_i\right(s\left)X_i\mathrm{d}{N}_i\right(t\left)\right\}=\mathcal{G}\left(s\right)f_S(s;x=1)P(X=1),E\left\{{\overline{Y}}_i\right(s\left)\mathrm{d}{N}_i\right(t\left)\right\}=\sum _{x=0}^1\mathcal{G}\left(s\right)f_S(s;x)P(X=x),r^{\left(1\right)}(\gamma ,s)=n\mathcal{G}\left(s\right){\mathcal{F}}_S(s;x=1)\exp \left(\gamma \right)P(X=1)$ and $r^{\left(0\right)}(\gamma ,s)=n\sum _{x=0}^1\mathcal{G}\left(s\right){\mathcal{F}}_S(s;x)\exp \left(\mathrm{x\gamma }\right)P(X=x)$, where $\mathcal{G}\left(s\right)=P(R_i\ge s)$ denotes the survival function for the independent random right censoring time R _i. Note that f _S(s;x) and ${\mathcal{F}}_S(s;x)$ are the density and survival functions for the surrogate progression‐free survival time S derived in Section 3.1, and they are functions of baseline intensities and treatment effects in the true three‐state illness‐death model as well as the parameters governing the assessment process. These enable one to evaluate how the bias in the estimator from the imputed data analysis changes according to these factors. In the Appendix, we report the results of a small investigation regarding the asymptotic behaviour of Cox regression coefficients of the treatment effect when the surrogate progression‐free survival data are used. For the scenarios studied, we find that there is generally a conservative bias, meaning the naive use of the surrogate progression‐free survival time underestimates the impact of treatment.

4. Simulations and analyses based on correct and misspecified models

Here, we report on simulation studies designed to empirically assess the bias and frequency properties of tests and confidence intervals in terms of power and coverage probabilities. Individuals were randomized to one of two treatment arms in a balanced fashion with equal probability. Given the treatment assignment, the times to progression or/and death were simulated based on the three‐state multiplicative model; T ₀₁ and T ₀₂ were simulated based on exponential distributions with rates λ ₀₁ exp(x β) and λ ₀₂ exp(x β), respectively, and if T ₀₁<T ₀₂, we simulate T ₁₂ according to an exponential distribution with rate λ ₁₂ exp(x β ₁₂). The values for the baseline intensities are set to satisfy the following constraints: (i) P(T ₀₁<T ₀₂∣X = 0) = λ ₀₁/(λ ₀₁+λ ₀₂)={0.6,0.8}; (ii) λ ₁₂/λ ₀₂=1.5; and (iii) π _A=P(T > C _A∣X = 0) = 0.20. We set β ₀₁=β ₀₂=β= log0.4 and β ₁₂= log1. The random right censoring time R _i is simulated by an exponential distribution with the rate set to achieve π _N={0.4,0.6}.

For the case with fixed assessment times, we assume K = 4 assessments occur over [0,C _A], where C _A=1, at times a _k=k C _A/K + ϵ _k, where ϵ _k∼N(0,σ _e) and k = 1,…,K. The random error ϵ _k is added to mimic the situation where some variation occurs around the evenly spaced assessment times in a protocol. We let σ _e={C _A/(6K),C _A/(20K)} for moderate and mild variation so that the chance of two consecutive assessment times would be realized in the reverse order is small. We plot (4) along with the KM estimates based on the actual progression‐free survival time (dashed line) and the surrogate progression‐free survival time (solid line) for a data set of n = 1000 individuals. The plots reveal a striking, familiar but misleading stepwise drop in the KM curves based on the surrogate event times as seen in 19, for example. This pattern becomes more distinctive as the variation around pre‐scheduled assessment times decreases (as shown in Figure 5(b)) and the risk of progression increases. In addition, the KM curves under right‐endpoint imputation sit above KM curves based on actual progression‐free survival event times and the theoretical curves.

Figure 5.

Kaplan–Meier curves estimated using right end point imputed progression‐free survival event times and actual progression‐free survival event times, contrasted against the theoretical survival function of progression‐free survival, for treatment and control groups. Number of fixed assessments K = 4 with a random normal noise ϵ _k∼N(0,σ _e). (β ₀₁=β ₀₂=β= log0.5,β ₁₂=0,P(T ₀₁<T ₀₂∣X = 0) = 0.80, λ ₁₂/λ ₀₂=1.5,π _A=0.2,π _N=0.4).

We also simulated random assessment times based on a Poisson process where the gap time between two consecutive assessments follows an exponential distribution with a rate α. For each parameter configuration, we calculated the sample size required to achieve 80% power for a Wald test at the 5% significance level under the Cox model. Three types of analysis were conducted on each simulated dataset for a given sample size including a Cox model for the true progression‐free survival time, a Cox model for the surrogate progression‐free survival time and a correct analysis based on the actual panel observation scheme using a time‐homogenous illness‐death model of Section 2. The first and third analyses yield consistent estimates of the treatment effect, but the first, of course, is not possible in practice. Under the assumption that the inspection process satisfies the sequentially missing at random condition 20 and is non‐informative, the partial likelihood has the form

$$\left\{\sum _{k=1}^{K_i}\mathrm{Pr}\left[Z\left(a_{\mathrm{ik}}\right)\mid Z\left(a_{i,k-1}\right),X_i)\right]\right\}{\left\{\sum _{j=0}^1\mathrm{Pr}\left[Z\left(\underset{i2}{\overset{-}{t}}\right)=j\mid Z\left(a_{i{K}_i}\right),X_i\right]{\lambda }_{j2}\left(t_{i2}\mid X_i\right)\right\}}^{I\left(T_{i2}<C_i\right)}.$$

This analysis can be implemented with the MSM package 26.

Table 1 reports the difference between the average estimated treatment effects and the true β (EBIAS) based on 1000 replications for each parameter configuration, the empirical standard error of the estimates (ESE), the average robust standard error (RSE) based on (15) and the empirical coverage probability of nominally 95% CIs for β (ECP). The proportion of replicates in which a test of H ₀:β = 0 is rejected under a two‐sided Wald test is also reported as the empirical power (EP). The Cox regression analysis of the actual event time and the multistate analysis yielded estimates with negligible empirical biases, but the naive Cox analysis based on the imputed event times gave biased estimates of treatment effect. The empirical coverage probabilities from all three analysis are very close to the nominal level. Although the multistate analysis based on panel data and the Cox model based on the true event time both yield consistent estimates, the former is associated with slightly larger standard errors and lower EP, reflecting the loss of information when event times are interval censored. More appreciable differences in EP are seen when comparing the naive Cox regression analysis with the parametric multistate analysis. The EP from the naive Cox regression analysis are below 80% in general and can be as low as 62.7%. The impact of the misspecification due to right‐endpoint imputation can therefore be quite substantial. A selection of results in Table 1 are graphically displayed in Figure A.1 in the Appendix, demonstrating good agreement between the empirical finite sample behaviour and results anticipated based on large sample theory.

Table 1.

Assessments arise from point process; number of simulations = 1000 for 80% statistical power.

			Cox model
			True PFS time†					Surrogate PFS time†					TH multistate analysis
α	P 1	N	EBIAS	ESE	RSE	ECP%	EP%	EBIAS	ESE	RSE	ECP%	EP%	EBIAS	ESE	RSE	ECP%	EP%
β = log(050),π N=40%
2.00	0.60	111	−0.003	0.263	0.253	94.0	78.5	0.059	0.284	0.277	93.9	62.7	−0.006	0.271	0.276	95.4	73.3
2.00	0.80	111	−0.007	0.257	0.252	94.7	80.9	0.078	0.272	0.268	93.0	64.6	−0.006	0.269	0.278	95.9	72.4
4.00	0.60	111	−0.003	0.263	0.253	94.0	78.5	0.028	0.284	0.269	94.4	69.4	−0.008	0.252	0.266	95.9	78.5
4.00	0.80	111	−0.007	0.257	0.252	94.7	80.9	0.037	0.287	0.277	93.6	67.1	−0.008	0.259	0.257	95.6	78.4
8.00	0.60	111	−0.003	0.263	0.253	94.0	78.5	0.008	0.280	0.263	94.2	73.6	−0.005	0.254	0.252	95.3	80.4
8.00	0.80	111	−0.007	0.257	0.252	94.7	80.9	0.006	0.280	0.267	94.2	74.3	−0.003	0.254	0.253	95.3	80.2
β = log(0.50),π N=60%
2.00	0.60	167	−0.004	0.253	0.253	95.1	80.0	0.039	0.272	0.265	93.2	69.8	−0.005	0.267	0.276	95.5	73.6
2.00	0.80	167	0.002	0.246	0.254	96.3	79.1	0.065	0.268	0.272	94.9	64.3	0.004	0.276	0.277	95.4	72.0
4.00	0.60	167	−0.004	0.253	0.253	95.1	80.0	0.014	0.269	0.267	94.9	73.2	−0.004	0.252	0.026	96.1	78.3
4.00	0.80	167	0.002	0.246	0.254	96.3	79.1	0.034	0.271	0.274	94.9	69.2	0.005	0.259	0.258	94.6	77.1
8.00	0.60	167	−0.004	0.253	0.253	95.1	80.0	0.003	0.269	0.266	94.3	74.3	0.009	0.249	0.253	95.5	79.4
8.00	0.80	167	0.002	0.246	0.254	96.3	79.1	0.018	0.265	0.271	95.0	72.2	−0.005	0.251	0.254	95.8	78.7
β = log(075),π N=40%
2.00	0.60	634	−0.002	0.104	0.103	94.1	82.0	0.023	0.111	0.114	94.9	65.4	−0.003	0.114	0.113	95.3	73.4
2.00	0.80	634	0.003	0.100	0.103	95.9	79.5	0.036	0.110	0.110	93.4	61.9	−0.006	0.112	0.113	94.1	71.7
4.00	0.60	634	−0.002	0.104	0.103	94.1	82.0	0.012	0.109	0.110	95.1	73.3	−0.002	0.102	0.105	95.8	78.4
4.00	0.80	634	0.003	0.100	0.103	95.9	79.5	0.021	0.111	0.113	95.0	66.6	−0.001	0.107	0.105	94.3	78.1
8.00	0.60	634	−0.002	0.104	0.103	94.1	82.0	0.001	0.107	0.107	94.7	76.4	−0.003	0.104	0.103	93.9	80.6
8.00	0.80	634	0.003	0.100	0.103	95.9	79.5	0.008	0.106	0.109	95.2	72.9	−0.003	0.105	0.104	94.8	78.8
β = log(075),π N=60%
2.00	0.60	952	−0.001	0.105	0.103	93.5	80.6	0.017	0.110	0.108	94.6	70.3	−0.006	0.111	0.112	94.7	76.0
2.00	0.80	952	−0.003	0.104	0.103	94.9	81.0	0.023	0.109	0.111	94.7	67.4	−0.004	0.114	0.112	94.9	75.1
4.00	0.60	952	−0.001	0.105	0.103	93.5	80.6	0.011	0.109	0.109	95.1	72.5	−0.003	0.103	0.105	95.5	79.3
4.00	0.80	952	−0.003	0.104	0.103	94.9	81.0	0.011	0.111	0.112	94.9	68.9	−0.004	0.105	0.108	94.3	79.9
8.00	0.60	952	−0.001	0.105	0.103	93.5	80.6	0.004	0.109	0.108	94.3	73.9	−0.000	0.103	0.103	94.8	79.6
8.00	0.80	952	−0.003	0.104	0.103	94.9	81.0	−0.001	0.108	0.110	94.5	74.7	−0.005	0.106	0.103	94.8	78.7

^†Both the true and surrogate progression‐free survival (PFS) times are subject to right censoring.β ₁₂= log1,P ₁=P(T ₀₁<T ₀₂∣X = 0) = λ ₀₁/(λ ₀₁+λ ₀₂),λ ₁₂/λ ₀₂=1.5,C _A=1,π _A=0.20.

5. Discussion

There has been much discussion about the use of composite endpoints in clinical trials 22, 23, and while views are somewhat divided, it is apparent that a clear interpretation of findings based on composite endpoints is difficult given the many factors influencing estimators. More discussion on use of progression‐free survival as a primary endpoint in oncology can be found in recent papers by 2, 3, 27, 28.

We have restricted attention to a idealized setting in which the treatment effect is the same for progression and death in order to simplify discussion and focus attention on the effects of intermittent assessments.

Frydman and Szarek 9 discuss the issue of interval‐censored progression times in the context of nonparametric estimation of progression‐free survival distribution. Boruvka and Cook 29 consider estimation and inference with the Cox model when two competing events are subject to different censoring schemes and interest lies in the transition intensity ratios. They point out that the present problem can be cast in this dual censoring framework, and hence, their methods could be utilized in this context. We have focused here, however, on the study of currently used methods to alert researchers to their limitations. Of particular importance is our finding that the standard practice of using the surrogate progression‐free survival time can lead to a substantial drop in the power of the trial when assessments are infrequent. To ensure trials are adequately powered, sample sizes should be increased to address the loss of power associated with the use of the surrogate event time S. One approach to this is to derive the sample size formula based on the three‐state illness‐death model addressing the interval‐censored progression times; piecewise constant baseline hazard function can be adopted to ease the calculation. Alternatively, one can simply adjust the sample size based on the misspecified model by deriving the limiting behaviour of the naive estimator and accommodating the bias and robust large sample variance in the calculations to ensure the power is maintained at the nominal level despite the misspecification.

These issues, while motivated by the setting of cancer clinical trials, arise in many other areas of health research involving joint consideration of non‐fatal events and death. Examples include osteoporosis trials in which elderly patients are at risk of both asymptomatic fractures (e.g. vertebral compression) and death where the former events are only detected by periodic bone scans, studies of cognitive impairment where periodic assessments may be scheduled and carried out until death or administrative censoring, among others.

Appendix A.

A.1. Asymptotic properties of treatment effect estimates from Cox regression on surrogate PFS

We set P(X = 1) = P(X = 0) = 0.5 and take the transition intensities in the illness‐death model to have a multiplicative form λ _jk(t) = λ _jk exp(x β). As before, values for the baseline intensities λ _jk are based on specifications for P(T ₀₁<T ₀₂∣X = 0) = λ ₀₁/(λ ₀₁+λ ₀₂),λ ₁₂/λ ₀₂ and π _A; see Section 3.1. We set β ₀₁=β ₀₂=β∈{log0.9, log0.75, log0.5} for mild, moderate and strong (beneficial) treatment effects. We let β ₁₂= log1 for no treatment effect on the intensity for progression to death transition.

Figure A.1 displays percent relative asymptotic bias (100(γ ^*−β)/β), asymptotic power associated with a Wald test of no treatment effect and the asymptotic coverage probability of a 95% CI when fitting a Cox model based on the surrogate progression‐free survival time under an ignorable random assessment process with intensity α(t) = α. The treatment effect is under‐estimated in general with the percent relative biases ranging from about 3% to 15%; see Figure 5. As expected, the magnitude of the bias increases as the rate of assessments decreases and as the competing risk of progression over death, P(T ₀₁<T ₀₂∣X = 0), increases. The size of the treatment effect itself seems to have very little impact on the magnitude of the percent relative bias. For each parameter configuration, we derived the sample size required to achieve an 80% power for a two‐sided Wald test of treatment effect at the 5% significance level under the Cox model for the progression‐free survival analysis. We then calculated the asymptotic power that one can actually obtain when the surrogate time is used for analysis. Figure A.1(c, d) shows that using the surrogate time can lead to an appreciable loss in power, which increases as the assessment rate decreases and the probability of progression increases. For example, when α = 2 and P(T ₀₁<T ₀₂∣X = 0) = 0.8, the power from the naive progression‐free survival analysis can drop to as low as 63% when the nominal level is 80%. The asymptotic coverage probability of the 95% CI, on the other hand, is quite robust to the model misspecification. Analogous results are found when the assessment times are fixed and, hence, are not reported here.

Figure A.1.

Asymptotic percent relative bias (100(γ ^*−β)/β), asymptotic power and asymptotic coverage probability of Cox regression coefficient of treatment effect from progression‐free survival analysis using imputed data; here, β ₀₁=β ₀₂=β={log0.5, log0.75, log0.9},β ₁₂= log1,λ ₁₂/λ ₀₂=1.5,π _A=0.2 and π _N=0.4.

Zeng, L. , Cook, R. J. , Wen, L. , and Boruvka, A. (2015) Bias in progression‐free survival analysis due to intermittent assessment of progression. Statist. Med., 34: 3181–3193. doi: 10.1002/sim.6529.