Estimation of Conditional Power for Cluster-Randomized Trials with Interval-Censored Endpoints

Summary:

Cluster-randomized trials (CRTs) of infectious disease preventions often yield correlated, interval-censored data: dependencies may exist between observations from the same cluster, and event occurrence may be assessed only at intermittent study visits. This data structure must be accounted for when conducting interim monitoring and futility assessment for CRTs. In this article, we propose a flexible framework for conditional power estimation when outcomes are correlated and interval-censored. Under the assumption that the survival times follow a shared frailty model, we first characterize the correspondence between the marginal and cluster-conditional survival functions, and then use this relationship to semiparametrically estimate the cluster-specific survival distributions from the available interim data. We incorporate assumptions about changes to the event process over the remainder of the trial—as well as estimates of the dependency among observations in the same cluster—to extend these survival curves through the end of the study. Based on these projected survival functions we generate correlated interval-censored observations, and then calculate the conditional power as the proportion of times (across multiple full-data generation steps) that the null hypothesis of no treatment effect is rejected. We evaluate the performance of the proposed method through extensive simulation studies, and illustrate its use on a large cluster-randomized HIV prevention trial.

1. Introduction

Cluster-randomized trials (CRTs) are well suited for the study of infectious disease prevention and intervention strategies (e.g., Pronyk et al., 2006; Lemaitre et al., 2009). By collectively randomizing groups of individuals to receive either the intervention or the standard of care, CRTs lessen the possibility of treatment contamination across randomization arms and allow investigators to capture both the direct and indirect effects of the intervention (Hayes and Moulton, 2017). However the data generated from these studies are often complexly structured. The “clustering effect” is a well-noted feature of CRTs: two individuals from the same cluster are more likely to be similar to one another than two individuals from different clusters. Furthermore, the outcome of interest in infectious disease CRTs is often a time-to-event outcome, such as time to HIV seroconversion. In the event that this outcome is asymptomatic or otherwise only observable via periodic examination (as is the case for HIV seroconversion), it is also interval-censored. Thus the design, monitoring, and analysis of these CRTs must account for correlated, interval-censored data.

Our focus here lies specifically on the issue of interim monitoring to permit early stopping for futility. Interim monitoring refers to the practice of evaluating a trial’s progress while the trial is ongoing, and is typically conducted through a series of interim statistical analyses (Proschan et al., 2006). The results of these analyses guide study decisions regarding sample size re-estimation, resource allocation, and early termination, and thus have important ethical and financial implications.

Two statistical frameworks are commonly used for determining if and when to stop a clinical trial early: group sequential testing (Pocock, 1977; O’Brien and Fleming, 1979; Pampallona and Tsiatis, 1994) and stochastic curtailment (Lan et al., 1982; Lachin, 2005). Group sequential methods calculate the test statistic of interest at each interim look, and stop the trial for either efficacy or futility if this test statistic crosses a pre-determined stopping boundary (Pampallona and Tsiatis, 1994). Stochastic curtailment approaches, on the other hand, involve calculating the conditional power, the conditional probability of rejecting the null hypothesis of no treatment effect at the scheduled end of the trial, given both the observed interim data and some conjecture about the remainder of the study. The trial stops for futility if this conditional power is low (Lachin, 2005).

Standard conditional power formulae for exponential family outcomes typically rely on the assumption that the test statistic, or some transformation thereof, is an asymptotically Brownian motion with linear drift; under this assumption, the conditional power may be expressed in terms of the standard normal cumulative distribution function (Lan et al., 1982; Lan and Wittes, 1988). This framework has since been adopted for the interim monitoring of trials with more complexly-structured outcomes, such as repeated measures or failure time data. In the repeated measures context, for example, Wu and Lan (1992) discussed the conditions under which a test statistic based on the linear mixed effects model could give rise to a discrete Brownian motion. Lin et al. (1999) addressed the monitoring of independent right-censored data, using martingale theory to establish that a broad class of weighted log-rank type statistics, possibly with covariate adjustment, converges to a Gaussian process. Martingale theory is not, however, available for interval-censored data due to the difficulty of defining an appropriate filtration. While the history of the associated counting process is a natural choice of filtration in the right-censored setting, this counting process is not well-defined for interval-censored data: when no exact failure times are observed, the value of the counting process at a particular time t, as well as the information generated by that process on the interval [0,t], may be unknown. As such, the formulae in Lin et al. (1999) cannot be easily extended to interval-censored outcomes. Simulation-based approaches for calculating conditional power with time-to-event endpoints have also been proposed, but these methods are currently limited to independent, right-censored data (Henderson et al., 1991).

Thus, while existing conditional power methods are able to address correlated data and right-censored time-to-event data separately, extensions to data that are both correlated and interval-censored are not available but are much needed. As a case in point, we consider the Botswana Combination Prevention Project (BCPP), a CRT evaluating the impact of combination HIV prevention on the 3-year cumulative incidence of HIV in 30 Botswanan communities (Makhema et al., 2019). To measure HIV incidence, a cohort of HIV-negative individuals from each of these communities was tested annually for the onset of infection, resulting in clustered, interval-censored data. When the study begin in 2013, one component of the combination prevention package was immediate antiretroviral therapy initiation for all HIV-positive individuals with high viral loads; the prevailing standard of care at the time was to initiate treatment only for those individuals with CD4 counts below 350 cells/mm³. However, during the course of the BCPP, the Botswana Ministry of Health updated the national treatment guidelines for HIV to a universal test-and-treat approach, recommending that all infected individuals—regardless of CD4 count or viral load level—receive antiretroviral therapy. This change likely reduced the magnitude of the expected intervention effect and raised concerns about the ability of the study to detect the updated effect. Interim monitoring of the BCPP for futility was thus particularly salient, but difficult given current interim monitoring and futility analysis methods (Makhema et al., 2019).

Motivated by this example, we propose a flexible simulation-based framework for estimating conditional power with correlated, interval-censored data. Our approach models the dependence between outcomes via a cluster-specific frailty and permits calculation of the conditional power under any assumed baseline hazard function and hazard ratio over the remainder of the study. It may also be used with any final analysis method. The statistical contributions of this paper are thus two-fold: it (i) presents the first conditional power method to explicitly consider correlated interval-censored data and (ii) does so in a manner that permits greater analytical flexibility than traditional formulae-based approaches.

The remainder of this paper is structured as follows. In Section 2, we describe in detail our proposed conditional power method. Section 3 examines the performance of this method across a range of data-generating mechanisms and clustering effects, as well as its sensitivity to misspecification of the dependence structure and to study design choices such as the use of matching, the number and size of the randomized clusters, and the frequency of the monitoring visits. In Section 4 we apply the proposed method to interim data patterned on the BCPP, and in Section 5 we conclude with a brief summary and discussion.

2. Methods

Suppose that we conduct a cluster-randomized trial of M independent clusters, with n_i individuals in cluster i and with ${\tau }_0^{(ij)}$ the calendar time of study entry for the jth individual in this cluster. Each of these individuals is then associated with a set of K planned monitoring times, at which the outcome of interest would be assessed. These monitoring times might be understood in terms of either calendar time (e.g., set to occur on a particular day each month) or internal time (e.g., set to occur annually post study enrollment); in either event, we transform these latent monitoring times so that ${\tau }_0^{(ij)}$ is the time origin and then denote them by {Y_ijk : k = 1,…,K}. Each individual is also associated with a set of K observation indicators, {R_ijk : k = 1,…,K}, with R_ijk = 1 if individual j in cluster i is present at the kth inspection. The set of observed monitoring times is then $\{Y_{ijk}^{*}=Y_{ijk}{R}_{ijk}:k=1,\dots ,K\}$. Let T_ij be the time to event for individual j in cluster i, also measured from ${\tau }_0^{(ij)}$. Note that, for asymptomatic events, we do not observe this failure time directly, but rather observe the interval (L_ij,U_ij], where L_ij is the last observed monitoring time at which individual j in cluster i tests negative for the presence of the event, and U_ij is the first observed monitoring time at which individual j in cluster i tests positive. Individuals who remain event-free at the last observed monitoring time are right-censored with observed interval $({\max }_k\{Y_{ijk}^{*}\},\infty )$. We assume throughout that the observed monitoring times ${\mathit{Y}}_{ij}^{*}=\{Y_{ijk}^{*}:k=1,\dots ,K\}$ are independent of the underlying time to infection, and that the underlying time to infection is independent of the calendar time of study entry.

Suppose that an interim analysis is scheduled for calendar time τ_I, and let $T_I^{(ij)}={\tau }_I-{\tau }_0^{(ij)}$ be the internal time of the interim analysis. Our proposed conditional power estimation procedure includes the following steps, as detailed in Sections 2.1 to 2.4 below (Figure 1). We (i) estimate the conditional survival function in each of the trial communities through time $T_I^{(i)}={\max }_j\{T_I^{(ij)}\}$ and then (ii) use these estimated curves—as well as assumptions regarding the subsequent event process and underlying dependence structure—to extend the survival functions through the remainder of follow-up. We then (iii) use a truncated inverse probability integral transform method to generate complete-trial observations from these extended curves and (iv) perform the prespecified final analysis on this complete-trial dataset. Finally, we (v) estimate the conditional power of the trial by repeating this data generation and analysis procedure multiple times, and by calculating the proportion of times that the null hypothesis of no intervention effect is rejected.

Figure 1.

An overview of the proposed conditional power calculation approach. The shaded boxes indicate the analysis pipeline, while the clear boxes indicate inputs to the pipeline that may either be user-specified or informed by the observed interim data.

2.1. Estimation of the Interim Survival Functions and Dependence Structure

Let X_i indicate cluster-level randomization to either intervention (X_i = 1) or standard of care (X_i = 0) at baseline. Our proposed estimation procedure is agnostic to whether this intervention assignment occurs entirely at random or whether it occurs conditional on some matching or stratification factors, provided that (i) individual outcomes are independent conditional on cluster membership and (ii) the hazard in cluster i can be written as

$$\lambda \left(t\mid X_i;{\eta }_i\right)=\lambda \left(t\mid X_i=0\right)\exp \left(\beta X_i+{\eta }_i\right),$$ (1)

where exp(η_i) is a cluster-specific frailty and λ(t|X_i = 0) is the baseline hazard function when ${\eta }_i\stackrel{set}{=}0$. For the purpose of exposition, we assume that these frailties follow a lognormal distribution with $\mathbb{E}\left({\eta }_i\right)=0$ and V ar(η_i) = σ², though our approach easily generalizes to other common frailty distributions (Web Appendix B).

Here we use the variance of the log-frailty terms, σ², to parametrize the CRT clustering effect, i.e., the extent of the within-cluster dependence that arises from cluster-level randomization. In CRTs with either binary or continuous (uncensored) endpoints, this dependence is often modeled via the intraclass correlation coefficient or ICC, which compares the outcome variability between clusters to the overall variability in the sample (Hayes and Moulton, 2017). However, the ICC is difficult to define and estimate when the outcome of interest is only partially observed due to censoring (see, for example, the discussion in Kalia et al. (2016)), and so an alternative measure of dependence, the coefficient of variation, k, may be used instead (Jahn-Eimermacher et al., 2013). The coefficient of variation captures the between-cluster variability in the cluster-specific hazard rates,

$$k=\frac{\sqrt{Var\left\{\lambda \left(t\mid X_i;{\eta }_i\right)\mid X_i\right\}}}{\mathbb{E}\left\{\lambda \left(t\mid X_i;{\eta }_i\right)\mid X_i\right\}}=\frac{\sqrt{Var\left\{\exp \left({\eta }_i\right)\mid X_i\right\}}}{\mathbb{E}\left\{\exp \left({\eta }_i\right)\mid X_i\right\}},$$

and admits a one-to-one relationship with σ² under the lognormal frailty model, namely that $k=k\left({\sigma }^2\right)=\sqrt{\exp \left({\sigma }^2\right)-1}$. Thus, the variance component of the shared frailty model in (1) captures the CRT clustering effect, while the cluster-specific frailties, exp(η_i), capture the resulting similarity in T_ij among individuals from the same cluster.

Throughout we will adopt the convention that λ(t|X_i; η_i) and S(t|X_i; η_i) denote the conditional hazard and survival functions within cluster i; that $\overline{\lambda }\left(t\mid X_i\right)$ and $\overline{S}\left(t\mid X_i\right)$ denote the hazard and survival functions marginalized over cluster membership; and that λ(t|X_i) and S(t|X_i) denote the conditional hazard and survival functions when ${\eta }_i\stackrel{set}{=}0$. We note that, under model (1), the conditional survival function in cluster i may equivalently be written as $S\left(t\mid X_i;{\eta }_i\right)=S{\left(t\mid X_i\right)}^{\exp \left({\eta }_i\right)}$ so that, provided that we are able to estimate both S(t|X_i) and exp(η_i), we may then take $\widehat{S}\left(t\mid X_i;{\eta }_i\right)=\widehat{S}{\left(t\mid X_i\right)}^{\exp \left({\widehat{\eta }}_i\right)}$ for t in $[0,T_I^{(i)}]$. Estimating the conditional survival functions in this fashion presents several notable advantages over stratified estimation of S(t|X_i; η_i) within each cluster. It (i) permits explicit characterization of the within-cluster dependence and coefficient of variation through σ² and exp(η₁),…, exp(η_M) while (ii) effectively leveraging information from all M clusters. This latter point is of particular importance when the outcome of interest is rare and the amount of information available in cluster i is otherwise small.

Thus estimation of S(t|X_i; η_i) under model (1) reduces to estimation of the common conditional survival function when η_i = 0, S(t|X_i), and of the M cluster-specific frailty terms, exp(η₁),…, exp(η_M). Methods for the semiparametric estimation of S(t|X_i) according to (1) are, however, generally limited, while methods for the nonparametric estimation of the marginal survival function, $\overline{S}\left(t\mid X_i\right)$, are readily available in standard statistical software (e.g., Turnbull, 1976; Wellner and Zhan, 1997). The relationship between these two curves depends on the assumed frailty distribution (see Web Appendix C for further discussion), but may—under the lognormal frailty model—be approximated by

$$\overline{S}\left(t\mid X_i\right)\approx S\left(t\mid X_i\right)\left[1+\frac{{\sigma }^2}{2}\log S\left(t\mid X_i\right)\left\{\log S\left(t\mid X_i\right)+1\right\}\right].$$ (2)

See Web Appendix A for the derivation of (2). A semiparametric estimator of S(t|X_i) is then given by the root of $g(x)=x\{1+\frac{{\sigma }^2}{2}\log x(\log x+1)\}-\overline{S}\left(t\mid X_i\right)$, which in turn requires reasonable estimators of $\overline{S}\left(t\mid X_i\right)$ and σ².

To that end, we nonparametrically estimate the marginal survival function in each intervention arm according to the independence data likelihood:${\prod }_{i=1}^M{\prod }_{j=1}^{n_i}\left\{\overline{S}\left(L_{ij}\mid X_i\right)-\overline{S}\left(U_{ij}\mid X_i\right)\right\}$. Although this likelihood is a misspecified version of the joint distribution—thus precluding valid inference on, for example, second-order terms—it still permits consistent estimation of the marginal parameters $\overline{S}\left(t\mid X_i=0\right)$ and $\overline{S}\left(t\mid X_i=1\right)$ (Chandler and Bate, 2007). Under nonparametric estimation, these survival functions are identifiable only up to the equivalence class of right-continuous, non-increasing functions defined by the values of $\{\overline{S}\left(L_{ij}\right):i=1,\dots ,M;j=1,\dots ,n_i\}\cup \left\{\overline{S}\left(U_{ij}\right):i=1,\dots ,M;j=1,\dots ,n_i\right\}$. While one common approach takes the nonparametric maximum likelihood estimator (NPMLE) to be the step function within this equivalence class, here we use linear interpolation to identify the marginal survival functions wherever the NPMLE is non-unique. This choice is made in recognition of the fact that—in settings where the censoring intervals are wide—the NPMLE may be unidentified on large regions of the support of T_ij; it may not be reasonable to assume that the survival function is flat during these inter-monitoring periods.

Recovering S(t|X_i) from $\overline{S}\left(t\mid X_i\right)$ also requires estimates of the variance term σ², while final estimation of S(t|X_i; η_i) requires estimates of the log frailty terms η = {η_i : i = 1,…,M}. Due to a lack of reliable software for frailty estimation with correlated interval-censored data, we first transform the interval-censored observations into right-censored data via midpoint imputation (Law and Brookmeyer, 1992) and then fit model (1) using a penalized partial likelihood approach (Therneau et al., 2003). This allows us to iteratively estimate both ${\widehat{\sigma }}^2$ and $\widehat{\mathit{\eta }}=\left\{{\widehat{\eta }}_i:i=1,\dots ,M\right\}$. As an aside, we note that—while midpoint imputation has been shown to produce biased coefficient estimates in Cox models fit to independent data—the simulation results in Section 3 suggest that any bias in the estimation of σ² and η has limited impact on the final conditional power estimates (Law and Brookmeyer, 1992).

2.2. Specification of the Event Process over the Remainder of the Trail

In order to calculate the conditional power, we also require some set of assumptions about the shape of the two conditional hazard functions when ${\eta }_i\stackrel{set}{=}0,\lambda \left(t\mid X_i=0\right)$ and λ(t|X_i = 1), over the remainder of the trial. Denote these hypothesized future values by $\tilde{\lambda }\left(t\mid X_i\right)$. Then under the assumption that there are no calendar trends in the event process of interest—so that the survival function S(t|X_i) is the same for all individuals with intervention assignment X_i, regardless of the time of study entry—the complete-trial survival function for the jth individual in the ith cluster, ${\tilde{S}}_{ij}\left(t\mid X_i;{\widehat{\eta }}_i\right)$, is given by

$$\{\begin{array}{ll}\widehat{S}{\left(t\mid X_i\right)}^{\exp \left({\widehat{\eta }}_i\right)},\hfill & t⩽T_I^{(ij)}\hfill \\ \widehat{S}{\left(T_I^{(ij)}\mid X_i\right)}^{\exp \left({\widehat{\eta }}_i\right)}\exp \left\{-{\int }_{T_I^{(ij)}}^t\tilde{\lambda }\left(u\mid X_i\right)\exp \left({\widehat{\eta }}_i\right)du\right\},\hfill & t>T_I^{(ij)}.\hfill \end{array}$$ (3)

The shape of these projected conditional hazard functions may be informed by scientific knowledge or determined by the investigator: our method permits any reasonable specification of $\tilde{\lambda }\left(t\mid X_i=0\right)$ and $\tilde{\lambda }\left(t\mid X_i=1\right)$. The projections may also be informed by the available interim data, and we detail one such approach to specifying $\tilde{\lambda }\left(t\mid X_i\right)$ below.

Within each trial arm, we approximate the conditional hazard function prior to interim via a step function with prespecified and equally-spaced knot points {ξ_s : s = 0,…,n}, where ξ₀ = 0 and ${\xi }_n=\min \{T_I^{(ij)}\}$. Let ${\lambda }_{X_i(s)}$ be the piecewise component on the interval [ξ_s−1, ξ_s) in the arm with intervention assignment X_i. Then we may write the conditional survival function at time ξ_s as $S\left({\xi }_s\mid X_i\right)=\exp \left\{-{\lambda }_{X_i(s)}\left({\xi }_s-{\xi }_{s-1}\right)\right\}S\left({\xi }_{s-1}\mid X_i\right)$, so that the corresponding component of the piecewise hazard is ${\lambda }_{X_i(s)}={\left({\xi }_s-{\xi }_{s-1}\right)}^{-1}\left\{\log S\left({\xi }_{s-1}\mid X_i\right)-\log S\left({\xi }_s\mid X_i\right)\right\}$. We then estimate ${\widehat{\lambda }}_{X_i(s)}$ by replacing S(ξ_s−1|X_i) and S(ξ_s|X_i) with the estimated values of the conditional survival functions when η_i = 0.

Let ${\Delta }_{{\lambda }_0}$ and ${\Delta }_{{\lambda }_1}$ be projected multiplicative changes to the conditional hazard function in the standard-of-care arm and in the intervention arm, respectively, over the remainder of the trial. We introduce these parameters to accommodate clinical settings in which the future event process is expected to differ systematically from that observed at interim, perhaps in light of delays in intervention roll-out or mid-trial modifications to the intervention. In the absence of additional information regarding the form of $\tilde{\lambda }\left(t\mid X_i\right)$, we make the final simplifying assumption that both $\tilde{\lambda }\left(t\mid X_i=0\right)$ and $\tilde{\lambda }\left(t\mid X_i=1\right)$ are constant, and then set $\tilde{\lambda }\left(t\mid X_i=0\right)={\overline{\lambda }}_0\cdot {\Delta }_{{\lambda }_0}$ and $\tilde{\lambda }\left(t\mid X_i=1\right)={\overline{\lambda }}_1\cdot {\Delta }_{{\lambda }_1}$, where ${\overline{\lambda }}_{X_i}=n^{-1}{\sum }_{s=1}^n{\widehat{\lambda }}_{X_i(s)}$.

2.3. Simulation of the End-of-Trial Data

In order to generate a final complete-trial dataset that captures both the observed interim data and the projected future event process, we must first identify the subset of study participants who are still at risk for the event of interest at interim. To that end, we note that, at the time of the interim analysis, there are three possible outcomes for individual j in cluster i: (i) they have been observed to have the event of interest, (ii) they have not had the event of interest but remain under active follow-up, or (iii) they have been lost to follow-up prior to time $T_I^{(ij)}$. Note that if missingness at the monitoring times is assumed to be intermittent rather than monotone, there may be no trial participants with interim outcome (iii). All individuals with interim outcomes (i) and (iii) thus have complete data at the time of the interim analysis; those with interim outcome (ii) remain at risk for the event of interest. For this last subset of individuals, we then simulate completed observations by generating both an underlying time to event and the subsequent observation process. The final complete-trial dataset then consists of the observed records from individuals with interim outcomes (i) and (iii) and the simulated records from individuals with interim outcome (ii).

2.3.1. Simulation of the underlying time to event.

Each individual j in cluster i with interim outcome (ii) is effectively right-censored at the time of the interim analysis, in that their observed censoring interval is (L_ij, ∞). Thus we wish to generate simulated times to event ${\tilde{T}}_{ij}$ according to the complete-trial survival function given in (3) and subject to the restriction that ${\tilde{T}}_{ij}>L_{ij}$. To do so, we first define the generalized inverse function ${\tilde{S}}_{ij}^{-1}\left(\omega \mid X_i;{\widehat{\eta }}_i\right)=\inf \{t\mid {\tilde{S}}_{ij}\left(t\mid X_i;{\widehat{\eta }}_i\right)⩽\omega \}$ and let $U\sim \mathrm{Uniform}(0,{\tilde{S}}_{ij}(L_{ij}\mid X_i;{\widehat{\eta }}_i))$. Then ${\tilde{T}}_{ij}\text{:=}{\tilde{S}}_{ij}^{-1}\left(U\mid X_i;{\widehat{\eta }}_i\right)$ follows the desired truncated distribution, with ${\tilde{T}}_{ij}>L_{ij}$. Web Appendix A provides formal justification for this truncated inverse probability integral transform method.

2.3.2. Simulation of the missingness and interval-censoring mechanisms.

We also require some method for simulating the subsequent observation process, which in turn requires generation of both the latent internal monitoring times, ${\tilde{Y}}_{ijk}$, and the observation indicators,${\tilde{R}}_{ijk}$. While we outline some possible modeling choices below, we note that any reasonable specification of the monitoring and missingness processes is permitted.

In CRTs and other clinical setting, monitoring times are typically planned a priori, though the actual visit dates of the individual study participants will inevitably vary in practice. Suppose that, at the time of the interim analysis, $0<K_{ij}^{\prime }<K$ of the K planned monitoring visits remain, with these subsequent visits scheduled for internal times ${\mathrm{\Upsilon }}_{ij,K-K_{ij}^{\prime }+1},\dots ,{\mathrm{\Upsilon }}_{ij,K}$. To incorporate uncertainty around the actual timing of these visits, we simulate the future monitoring times ${\tilde{Y}}_{ijk}\sim \mathrm{Uniform}\left({\mathrm{\Upsilon }}_{ij,k}-\delta ,{\mathrm{\Upsilon }}_{ij,k}+\delta \right)$ for some prespecified δ > 0 and for $k=K-K_{ij}^{\prime }+1,\dots ,K$. We also set ${\tilde{Y}}_{ij,K+1}:=\infty$.

Missingness at these visits may be modeled as either intermittent or monotone. In the case of the former, the future observation indicators $\{R_{ijk}:k=K-K_{ij}^{\prime }+1,\dots ,K\}$ are a sequence of Bernoulli random variables, with the probability of being observed at visit $k,{\pi }_{ijk}^R$, optionally depending on the available baseline covariates and the prior history of missingness. ${\pi }_{ijk}^R$ may then be estimated using a generalized linear mixed model with individual- and cluster-specific frailties and a logit link and we simulate ${\tilde{R}}_{ijk}\sim \mathrm{Bernoulli}({\widehat{\pi }}_{ijk}^R)$.

In the case of the latter, the observation indicators for individual j in cluster i represent a coarsening of some latent loss to follow-up time, C_ij, with $R_{ijk}=I\left(Y_{ijk}⩽C_{ij}\right)$. Simulation of the future observation indicators $\{R_{ijk}:k=K-K_{ij}^{\prime }+1,\dots ,K\}$ thus reduces to the simulation and appropriate transformation of this C_ij. To that end, we first assume that C_ij ~ Exp(λ_C), where λ_C is the rate of study attrition. To estimate λ_C from the observed data, we additionally note that the C_ij are, in effect, interval-censored. To see this, consider again the three possible outcomes for individual j in cluster i at interim. Under interim outcome (i), the censoring interval for C_ij is simply [U_ij, ∞), while under interim outcome (ii) the censoring interval is [L_ij, ∞). Under interim outcome (iii), let $Y_{ij{K}_{ij}^{\prime \prime }}$ be the internal time of the last observed visit and ${\mathrm{\Upsilon }}_{ij,K_{ij}^{\prime \prime }+1}$ be the (planned) internal time of the first missed visit. Then the corresponding censoring interval is $[Y_{ij{K}_{ij}^{\prime \prime }},{\mathrm{\Upsilon }}_{ij,K_{ij}^{\prime \prime }+1})$. Regardless of the outcome scenario, let $[L_{ij}^{*},U_{ij}^{*})$ be the observed censoring interval for C_ij. We estimate λ_C by maximizing the corresponding independence likelihood and then simulate ${\tilde{C}}_{ij}\sim \mathrm{Exp}({\widehat{\lambda }}_C)$ subject to ${\tilde{C}}_{ij}>L_{ij}^{*}$ using the truncated inverse probability transform. The set of corresponding observation indicators is simply $\{{\tilde{R}}_{ijk}=I({\tilde{Y}}_{ijk}⩽{\tilde{C}}_{ij}):k=K-K_{ij}^{\prime }+1,\dots ,K\}$.

Regardless of the assumed missingness mechanism, we set the observation indicator for the final visit time ${\tilde{Y}}_{ij,K+1}$ to be ${\tilde{R}}_{ij,K+1}\text{:=}\text{1}$. Then the simulated observation process post interim analysis is given by $\{{\tilde{Y}}_{ijk}^{*}={\tilde{Y}}_{ijk}{\tilde{R}}_{ijk}:k=K-K_{ij}^{\prime }+1,\dots ,K+1\}$.

2.3.3. Generation of the final complete-trial observations.

For each study participant with interim outcome (ii), we first simulate both the underlying time to event, ${\tilde{T}}_{ij}$, and the subsequent observation process,$\{{\tilde{Y}}_{ijk}^{*}:k=K-K_{ij}^{\prime }+1,\dots ,K+1\}$. Let ${\tilde{\mathit{Y}}}_{ij}^{*}$ represent the corresponding full-trial observation process, comprised of both the observed and simulated inspections: ${\tilde{\mathit{Y}}}_{ij}^{*}=\{Y_{ijk}^{*}:k=1,\dots ,K-K_{ij}^{\prime }\}\cup \{{\tilde{Y}}_{ijk}^{*}:k=K-K_{ij}^{\prime }+1,\dots ,K+1\}$. Then the final simulated observation is $\left({\tilde{L}}_{ij},{\tilde{U}}_{ij}\right]$, where ${\tilde{L}}_{ij}=\max \{{\tilde{Y}}_{ijk}^{*}:{\tilde{Y}}_{ijk}^{*}\in {\tilde{\mathit{Y}}}_{ij}^{*}\text{and}{\tilde{Y}}_{ijk}^{*}<{\tilde{T}}_{ij}\}$ and ${\tilde{U}}_{ij}=\min \{{\tilde{Y}}_{ijk}^{*}:{\tilde{Y}}_{ijk}^{*}\in {\tilde{\mathit{Y}}}_{ij}^{*}\text{and}{\tilde{Y}}_{ijk}^{*}⩾{\tilde{T}}_{ij}\}$. A diagram further illustrating the process of generating complete-trial observations is provided in Figure 2.

Figure 2.

Sample complete-trial generation process under monotone missingness for four members of cluster i, with the observed data rendered in black and the projected data rendered in blue. The final interval-censored observations are given on the right.

2.4. Calculation of the Conditional Power

For a given specification of $\tilde{\lambda }\left(t\mid X_i\right)$ and of the monitoring and missingness processes, we generate C complete-trial datasets with correlated interval-censored outcomes, and then conduct the prespecified final analysis on each of these simulated datasets; potential analysis methods for correlated interval-censored data include mixed-effects accelerated failure time modeling (e.g., Komárek and Lesaffre, 2007) and randomization-based inference (Wang and De Gruttola, 2017). We then calculate the conditional power as the proportion of the C p-values that are significant at the desired α level.

2.5. Sensitivity Analyses

In order to better guide study termination decisions, we recommend that investigators conduct a series of sensitivity analyses for the conditional power. These sensitivity analyses might take the following form. To account for the presence of estimation or investigator uncertainty, the analyst might first construct confidence intervals for each estimated interim parameter (e.g., the conditional hazard functions) or plausible intervals for each projected component of the event process, and then recalculate the conditional power at various values across this plausible region. This produces a range of conditional power values consistent with the observed data and/or the investigator’s uncertainty about the remainder of the trial. Other potential sensitivity analyses include calculating the conditional power under the null trend or under the minimum clinically-meaningful effect. Finally, in order to evaluate the impact of clustering on the final probability of study success, investigators may also wish to calculate the conditional power under an array of plausible coefficients of variation, k = k(σ²), and associated log-frailty terms, η_i ~ N(0, σ²).

3. Simulation Study

We conducted a series of simulation studies to examine the performance of our proposed conditional power method, as well as to characterize its robustness to both CRT design considerations and the specification of the frailty distribution.

3.1. Simulation Settings and Evaluation Procedure

3.1.1. Data generating mechanism.

We simulated data representative of both matched and unmatched CRTs designs, with M = 30 clusters and n_i ∼ Uniform(250,350) individuals in cluster i. For each enrolled cluster, we first generated an independent log-frailty term, η_i ~ N(0, σ²), and took the pre-randomization hazard function to be λ(t; η_i) := λ₀ exp(η_i). In the unmatched setting, we then assigned the intervention (X_i = 1) to half of these clusters at random, i.e., independent of the underlying λ(t; η_i), and generated the calendar times of study entry according to ${\tau }_0^{(ij)}\sim \mathrm{Uniform}(2i-2,2i+2)$, reflecting staggered study enrollment. In the matched setting, on the other hand, we first ranked the clusters according to their pre-randomization hazard functions before forming matched pair g (g = 1,…,15) from the clusters with conditional hazard functions λ(t; η_(2g−1)) and λ(t; η_(2g)); we then randomized intervention assignment conditional on matched pair membership. We also assumed that the calendar times of study enrollment followed the same distribution within each matched pair, so that ${\tau }_0^{(ij)}\sim \mathrm{Uniform}(4g-4,4g)$ for all individuals in pair g.

Regardless of the assumed study design, the true time to event for each participant was then generated according to model (1) and was subject to monitoring at K = 4 annual inspections with Y_ijk ~ Uniform(52k − 4,52k + 4). Missingness at these visits was modeled as monotone with C_ij ~ Exp(0.002), corresponding to an overall loss to follow-up rate of approximately 10% per year. We interval censored each time to event as a function of this simulated observation process, and then created the interim analysis datasets by truncating the observations at calendar time ${\tau }_I=\max \{Y_{ij2}+{\tau }_0^{(ij)}\}$. As a consequence of the staggered study entry, participants still under active follow-up at the time of the interim analysis had either two and three observed monitoring visits.

3.1.2. Simulation settings and structure.

We considered two values for the conditional baseline hazard function, representing settings in which the event of interest was rare and heavily administratively censored (λ₀ = 0.001) versus common and heavily interval-censored (λ₀ = 0.01), and varied the conditional intervention effect β from [−1, 1] at 0.1 intervals. We also took σ² ∈ {0, 0.06, 0.22}, corresponding to coefficients of variation of k ∈ {0, 0.25, 0.50} (and thus an absent, mild, and moderate clustering effect), respectively. Our primary simulation study consisted of 1000 simulation replicates for each combination of the baseline hazard, intervention effect, and log-frailty variance. We also considered a series of more focused simulation experiments in which we varied the number and size of the randomized clusters, the frequency of the monitoring visits, and the data-generating frailty distribution under the unmatched CRT setting. We took ${\Delta }_{{\lambda }_0}={\Delta }_{{\lambda }_1}=1$ throughout.

3.1.3. Analysis method.

For each generated interim dataset, we evaluated the conditional power by testing the null hypothesis of no intervention effect across 500 sets of projected complete-trial data using a permutation test (Wang and De Gruttola, 2017). In the matched CRT setting, we took the permutation test statistic to be the sum of the pair-specific cumulative incidence differences, $T^{obs}={\sum }_{g=1}^{15}({\widehat{\Lambda }}_{1g}-{\widehat{\Lambda }}_{0g})$, while in the unmatched setting we instead considered all possible pairwise intervention/standard of care comparisons. We then constructed the permutation null by randomizing the intervention assignment—either within matched pair or across all clusters—and took the conditional power to be the proportion of the resulting permutation p-values that reached significance at an α = 0.05 level.

3.1.4. Performance evaluation.

To evaluate the overall performance of our conditional power method, we compared the mean conditional power across the 1000 simulation replicates to the simulated power for that setting. While we note that the conditional and unconditional power of a trial are not directly comparable, the mean of the empirical conditional power distribution estimates the trial power. In particular, if μ_r is the empirical probability measure for the observed interim data, $\mathcal{I}$, across the r = 1000 simulation replicates and μ is the true probability measure for $\mathcal{I}$, then $\int \mathbb{P}(p(\mathcal{D})⩽\alpha \mid \mathcal{I})d{\mu }_r\approx \int \mathbb{P}(p(\mathcal{D})⩽\alpha \mid \mathcal{I})d\mu =\mathbb{P}(p(\mathcal{D})⩽\alpha )$ for any p-value $p(\mathcal{D})$ derived from the complete-trial data,$\mathcal{D}$. Thus, if our conditional power method performs as designed, we would expect the mean conditional power to approximate the simulated power for each simulation setting.

We also calculated the conditional power under the modification (unachievable in practice) in which $\tilde{\lambda }\left(t\mid X_i=0\right):={\tilde{\lambda }}_0$, $\tilde{\lambda }\left(t\mid X_i=1\right):={\tilde{\lambda }}_1,{\sigma }^2$, and η are all set to their data-generating values. This allowed us to isolate whether discrepancies between the mean conditional power and the simulated power were the result of the projection procedure, the specification and estimation of the parameter vector, or both.

3.2. Simulation Results

Here we present results for two CRT settings: one in which the event incidence is low and the clustering effect mild (λ₀ = 0.001; σ² = 0.06), as this most closely resembles the motivating BCPP example, and the other in which the event incidence is high and the clustering effect moderate (λ₀ = 0.01; σ² = 0.22). Additional simulation results for other incidence/clustering effect combinations and design parameters are presented in Web Appendices D and E.

In both of these CRT settings, we find that our proposed conditional power method successfully replicated the salient features of the original complete-trial datasets under both the matched and unmatched trial designs (see Web Figures S5–S6 and Web Table S2, which present select results for β ∈ {−0.2, 0, 0.2}); in so doing, our procedure was able to capture both general trends in study conditional power across the simulation settings (Figure 3; Web Figure S7), as well as specific fluctuations in evidence strength across the individual interim datasets (Figure 4; Web Figures S3, S4, and S8). When ${\tilde{\lambda }}_0$, ${\tilde{\lambda }}_1$, σ², and η were all set to their data-generating values, the mean conditional power closely approximated the simulated study power for all baseline hazard, intervention effect, and dependence settings. This close correspondence confirms that—absent any estimation error in either the event process or the clustering effect—the resulting conditional power estimates reasonably capture (on average) the underlying study futility. When ${\tilde{\lambda }}_0$, ${\tilde{\lambda }}_1$, σ², and η were instead estimated as in Sections 2.1 and 2.2—and as might be done in practice—the mean conditional power continued to match the simulated trial power.

Figure 3.

Power and conditional power results as a function of the true, data-generating baseline hazard, intervention effect, and log-frailty variance in the generated interim data. Each data point represents the mean estimated power (conditional power) across 1000 simulation replicates.

Figure 4.

Accuracy rates of the interim futility classifications based on a futility threshold of 20% conditional power and a final significance level of α = 0.05. We consider two CRT settings: one in which the event rate is low and clustering effect mild (λ₀ = 0.001; σ² = 0.06) and the other in which the event rate is high and clustering effect moderate (λ₀ = 0.01; σ² = 0.22).

In addition to capturing aggregate trends in study futility, our proposed method produced individual conditional power estimates that correlated in a meaningful way with the significance of the individual completed studies (Figure 4; Web Figures S3, S4, and S8). Assuming that the decision to stop (or not stop) a trial early is based on a futility threshold of 20% conditional power, 9.3% of the 42,000 simulated matched CRTs were classified as futile at interim (i.e., would have been stopped early on the basis of low estimated conditional power), with 88.6% of these trials having been classified correctly (i.e., ultimately resulting in final p-values above α = 0.05); of the 38,085 matched trials that were classified as not futile at interim, 98.3% had final p-values below α = 0.05. Among the simulated unmatched CRTs, these true negative and true positive rates were 98.1% and 96.0%, respectively.

The final correct classification rate over all 84,000 simulated trials across the two featured CRT settings was 96.9%, with almost all of these classification errors attributable to continuing studies that ultimately proved futile. This elevated type II error rate is a reflection of the modest inflation of the mean conditional power estimates relative to the study power observed in those simulation settings with near-null intervention effects (Figure 3). This inflation is likely the result of a floor effect: even when data are representative of a null intervention effect, the observed conditional hazard functions ${\widehat{\overline{\lambda }}}_{X_i}$ will inevitably differ from one another slightly, and the resulting conditional power estimates will reflect this spurious effect.

3.3. Sensitivity to the Study Design and Misspecification of the Frailty Distribution

Our proposed method continued to produce reasonable conditional power estimates across several modifications to the CRT study design, including variations in the number and size of the randomized clusters (Web Table S3) and the frequency of the monitoring visits (Web Table S4). Its performance was also robust to misspecification of the frailty distribution: when the true frailties were gamma-distributed, the conditional power estimates derived under the misspecified lognormal model were nearly identical to those under the correctly-specified gamma model (Web Table S5).

4. The Botswana Combination Prevention Project

The Botswana Combination Prevention Project (BCPP) was a pair-matched CRT designed to evaluate the impact of combination HIV prevention strategies on the population-level three-year cumulative incidence of HIV (Makhema et al., 2019). Thirty communities in Botswana were pair-matched on the basis of size, pre-existing health services, population age structure and geographic location, with one community in each pair randomized to receive combination HIV prevention and the other to receive an enhanced standard of care. An incidence cohort of HIV-negative individuals identified from a 20% random sample from each community was tested for seroconversion at each of three annual visits; an interim analysis was planned following the completion of all one-year visits.

At the beginning of the study in 2013, the combination prevention package included scaled-up HIV testing and counseling services, linkage-to-care support, enhanced mother-to-child transmission prevention efforts, male circumcision campaigns, and extension of antiretroviral therapy to those infected individuals with high viral load levels. The enhanced standard of care included higher testing coverage (due to the annual testing of the incidence cohort) and improved technical support for data management, but otherwise reflected the contemporary standard practice. However, while the trial was ongoing the national HIV treatment guidelines changed: the Botswana Ministry of Health recommended in 2016 that all HIV-positive patients, regardless of CD4 count or viral load levels, initiate antiretroviral therapy (Makhema et al., 2019). This change made the care administered to the control communities more similar to that in the intervention communities, raising concerns that the anticipated intervention effect might be reduced and that the trial might be futile as a result.

Actual interim data from the BCPP remain confidential. As such, we used our proposed method to conduct a futility assessment on a simulated interim dataset, generated by applying an agent-based epidemic model to a dynamic network of simulated sexual partnerships (Wang et al., 2014). Both the sexual network and epidemic models were developed to project the intervention effect during the design phase of the BCPP, and the inputs to these models were calibrated to resemble the actual study conditions (see Web Appendix F for details).

4.1. Futility Assessment Using the Network-Generated Data

The simulated BCPP trial followed an incidence cohort of 10,465 individuals across the 30 study communities: 5,225 of these individuals belonged to combination prevention communities, and 5,240 to enhanced standard-of-care communities. By the time of the interim analysis, 35 individuals in the combination prevention clusters and 50 individuals in the enhanced standard-of-care clusters had seroconverted, corresponding to cluster-conditional incidence rates of 0.0045 and 0.0064 events per person-year, respectively (estimated conditional hazard ratio: 0.699). Assuming no change to these trends, the conditional power of the simulated BCPP was estimated to be 0.630. However, if the observed intervention effect were to diminish by 10% over the remainder of the trial in response to the new Ministry of Health treatment guidelines (reducing the hazard ratio from 0.699 to 0.769), the estimated conditional power would drop to 0.498. Reductions in the underlying baseline hazard of seroconversion, as might result from the mid-trial adoption of universal antiretroviral therapy initiation, also led to a modest drop in the estimated conditional power (Table 1). For a full summary of the estimated conditional power results, as well as information regarding the projected total person-time of follow-up and number of HIV seroconversions at study conclusion across the simulated complete-trial datasets, see Table 1. The full analysis was executed in R on a single core of the Harvard Medical School O2 cluster and took approximately 1.5 hours to complete, for an average time of 9 minutes per conditional power estimate.

Table 1.

Conditional power and complete-trial projection results for interim data that were patterned on the Botswana Combination Prevention Project (BCPP). These data were generated from an agent-based epidemic model that was spread on a simulated sexual contact network, with inputs informed by BCPP data when available. Number of events, full trial conditional hazard ratios, and person-time are summarized as the mean (and range or standard deviation) across 500 generated end-of-trial datasets; the conditional power is reported as the proportion of those 500 datasets in which the null hypothesis of no effect was rejected.

${\mathrm{\Delta }}_{{\lambda }_0}$	${\tilde{\mathrm{\lambda }}}_0$	${\mathrm{\Delta }}_{{\mathrm{\lambda }}_0}/{\mathrm{\Delta }}_{{\mathrm{\lambda }}_1}$	${\tilde{\mathrm{\lambda }}}_1/{\tilde{\mathrm{\lambda }}}_0$	Number of Events			Person-Time (person-years)		Conditional Power
				Intervention	No Intervention	Full HR	Intervention	No Intervention
0.9	5.8 × 10−3	0.8	0.559	57.1 (43, 72)	88.8 (71, 112)	0.639 (0.099)	761,751.9 (1,607.2)	759,171.5 (1,540.7)	0.828
		0.9	0.629	59.9 (45, 75)	89.5 (75, 110)	0.667 (0.091)	761,580.6 (1,630.2)	759,106.1 (1,718.1)	0.752
		1.0	0.699	62.9 (49, 77)	89.1 (71, 110)	0.712 (0.102)	761,513.6 (1,623.1)	759,049.1 (1,654.8)	0.580
		1.1	0.769	65.1 (50, 81)	89.4 (73, 106)	0.736 (0.102)	761,291.8 (1,602.7)	759,045.7 (1,528.8)	0.486
		1.2	0.839	68.3 (53, 87)	89.4 (74, 110)	0.775 (0.108)	761,408.9 (1,623.1)	759,022.4 (1,593.3)	0.340
1.0	6.4 × 10−3	0.8	0.559	59.7 (43, 76)	93.8 (74, 114)	0.631 (0.088)	761,556.7 (1,664.8)	758,882.7 (1,650.2)	0.856
		0.9	0.629	63.0 (48, 80)	93.7 (77, 115)	0.670 (0.099)	761,392.8 (1,612.0)	758,846.0 (1,610.6)	0.730
		1.0	0.699	65.3 (49, 83)	93.4 (75, 112)	0.699 (0.100)	761,353.0 (1,653.0)	759,017.2 (1,642.2)	0.630
		1.1	0.769	68.7 (54, 87)	94.1 (74, 117)	0.736 (0.106)	761,167.8 (1,576.5)	759,018.2 (1,666.6)	0.498
		1.2	0.839	71.5 (56, 88)	93.5 (72, 115)	0.773 (0.116)	761,037.9 (1,718.4)	758,851.2 (1,520.3)	0.364

5. Discussion

We have proposed a flexible framework for calculating the conditional power of CRTs with interval-censored endpoints. Our approach permits any assumed form for the conditional hazard functions over the remainder of the study, and may be used with any final hypothesis test of interest. This second feature presents a notable advantage over standard conditional power formulae, which are typically specific to a given test statistic or class of test statistics. Extensive simulation studies demonstrated that the proposed method produces reasonable conditional power estimates across an array of data-generating models and a wide range of study design parameters, and that it is robust to misspecification of the frailty distribution.

We observed mild inflation in the conditional power estimates in those settings with low study power, most likely due to a floor effect on the estimation of the intervention effect. However, the futility bound for clinical trials with conditional-power-based interim monitoring is often set between 10% and 30% (Zhang et al., 2017), with recent examples including the oncology trial LUME-Lung 2, for which the futility threshold was set at 20% (Lesaffre et al., 2017), and a psychiatric trial of maintenance treatment of bipolar I disorder, for which the futility threshold was set at 30% (Mahableshwarkar et al., 2017). Under these more conservative futility thresholds, our conditional power procedure remains able to accurately discriminate between futile and non-futile trials at interim, even in the presence of this mild conditional power inflation.

In order to estimate the conditional survival functions S(t|X_i; η_i), we adopted the lognormal shared frailty model given in (1). Few computational methods permit stable fitting of (1) to large clusters of interval-censored observations, and semiparametric estimation of this model with correlated interval-censored data remains an open area of research. As such, we used midpoint imputation to transform the interval-censored observations into right-censored observations, from which we then estimated the log-frailty variance and individual frailty terms. However, midpoint imputation has been shown to produce biased coefficient estimates in (1), with the magnitude of the bias increasing with the width of the censoring intervals. For this reason, we exploited the relationship between the marginal and conditional survival functions to obtain a semiparametric estimator for S(t|X_i), rather than relying on a midpoint-imputed estimate. Further work is needed to develop stable procedures for fitting frailty models directly to correlated interval-censored observations.

This point is particularly salient when we consider extensions of our conditional power procedure to the stepped-wedge CRT (SW-CRT) context, in which all clusters begin follow-up in the standard-of-care arm before crossing over to the intervention arm in a randomized order (Copas et al., 2015). In light of this cross-over event, intervention assignment is effectively a time-dependent covariate, such that we might replace the frailty model in (1) by λ(t|X_ij(t); η_i) = λ₀(t)exp{βX_ij(t)+η_i}, η_i ~ N(0, σ²). Provided that we are able to estimate each component of this model, our conditional power framework may be readily applied to the SW-CRT setting as well.

The main focus of this article has been on proposing a method for calculating point estimates of the conditional power. Clinical decision-making would be further aided by having some tool to characterize uncertainty in these estimates. One natural choice for estimating this variability is the bootstrap 95% confidence interval. However, there are several complications that arise when performing bootstrap resampling on correlated, interval-censored observations, particularly when the outcome of interest is both rare and interval-censored. CRTs typically randomize only a small number of communities, so that bootstrap resampling at the cluster level is unlikely to provide a reasonable approximation to the sampling distribution. Resampling at the individual level conditional on cluster membership is more amenable to a small M setting, but may produce clusters in which no event has occurred when the outcome of interest is rare; this in turn limits the types of test statistics that can be computed on the bootstrapped interim data. In light of these challenges, we instead recommend conducting sensitivity analyses to assess the conditional power across a range of projections that are compatible with the observed data; these analyses should provide the additional context and uncertainty quantification needed for clinicians to make appropriate and informed trial termination decisions at the futility boundary.

Data Availability Statement

The simulated Botswana Combination Prevention Project data used to illustrate the approach proposed in this paper are openly available in the GitHub repository kaitlyncook/cp-crt at https://doi.org/10.5281/zenodo.3968791 (Cook and Wang, 2020).