A Log Rank Type Test in Observational Survival Studies with Stratified Sampling

Abstract

In randomized clinical trials, the log rank test is often used to test the null hypothesis of the equality of treatment-specific survival distributions. In observational studies, however, the ordinary log rank test is no longer guaranteed to be valid. In such studies we must be cautious about potential confounders; that is, the covariates that affect both the treatment assignment and the survival distribution. In this paper, two cases were considered: the first is when it is believed that all the potential confounders are captured in the primary database, and the second case where a substudy is conducted to capture additional confounding covariates. We generalize the augmented inverse probability weighted complete case (AIPWCC) estimators for treatment-specific survival distribution proposed in Bai et al. (2013) and develop the log rank type test in both cases. The consistency and double robustness of the proposed test statistics are shown in simulation studies. These statistics are then applied to the data from the observational study that motivated this research.

1 Introduction

The ASCERT study, funded by the National Heart Lung and Blood Institute, was an observational study of patients with either two-vessel or three-vessel coronary artery disease. The patients in the study were treated either by surgical revascularization (coronary artery bypass grafting; CABG) or catheter-based revascularization (percutaneous coronary intervention; PCI). The goal of the ASCERT study is to compare these two treatment options for patients with coronary artery disease (Weintraub et al., 2012). Not all patients were followed until the endpoint of interest (all-cause mortality); accordingly, survival outcomes for these patients were censored. Moreover, a substudy was conducted to collect additional covariate information using a stratified random sample of patients from the main ASCERT study. Bai et al. (2013) used semiparametric methods to provide doubly-robust estimators of treatment-specific survival distributions using the data from the main study as well as taking advantage of the additional potential confounders from the substudy. Besides estimating treatment-specific survival distributions, the comparison of treatment-specific survival distributions is also of importance. In a randomized controlled clinical trial where the covariate distribution is balanced among different treatment groups, the log rank test is most commonly used to test the null hypothesis of no difference between the treatment-specific survival distributions. In an observational study, however, the traditional log rank test is no longer valid due to possible confounding.

Xie and Liu (2005) proposed an inverse probability weighted log rank test to adjust for such confounding. They computed a weighted version of log rank statistics by substituting inverse probability weighted number of subject at risk and number of subject that die in each group. This estimator would provide valid inference if the propensity model is consistently estimated. Zhang and Schaubel (2012) compared treatment groups in term of the difference in restricted mean lifetime, which they referred as the average causal effect. This is a semiparametric estimator with a doubly-robust property. However, its consistency relies on the underlying assumption that the censoring time and survival time are independent conditional on the treatment. If the covariates also have effect on censoring which may likely be the case in observational studies, this method might be subject to bias.

In this paper, we propose a log rank type test statistic to compare treatment groups. With a nonparametric bootstrap estimator of the denominator, the resulting test statistic will be doubly-robust. Moreover, we generalize the test statistic into the case where a substudy is to be conducted to collect additional covariates to account for possible residual confounding.

In the next section, we derive the log rank test statistic using the data from the main study which is appropriate when the belief is that the the collected covariates are enough to adjust for all potential confounding. In Section 3, we develop test statistics to deal with the case when we further conduct a stratified sampling substudy to collect additional potential confounding covariates. The performance of the proposed test statistics are demonstrated by simulation studies in Section 4. We then apply our method to the motivating example of ASCERT study to test for the treatment differences in CABG and PCI in Section 5. The paper ends with discussion and conclusion in Section 6.

2 Test Statistic for Main Study when all Potential Confounders are Captured

As in the motivating example of the ASCERT study, we are interested in the comparison of survival distributions in an observational study. In this article, we use the same notation as in Bai et al. (2013). We denote the two treatments used in the study as treatment 1 and treatment 0. For each individual i in the main study, we define the potential outcomes (X_i, $T_i^{(0)},T_i^{(1)},C_i^{(0)},C_i^{(1)}$), i = 1, …, N, where X_i denotes a vector of baseline covariates, $T_i^{(j)}$ denotes the potential survival time and $C_i^{(j)}$ denotes the potential censoring time if individual i were given treatment (possibly contrary to fact) j, for j = 0, 1. In the case of the ASCERT study, all patients were followed from the date they entered the study until the time of data analysis and their survival status was fully ascertained during this period. Consequently, the potential censoring time was the time from entry into study until time of analysis which would be the same under both treatments. Therefore we have $C_i^{(1)}=C_i^{(0)}=C_i$ for i = 1, …, N. The treatment-specific survival distributions are defined as S^(j)(u) = P(T^(j) ≥ u) for j = 0, 1 and the main focus of this paper is to compare the overall survival distributions S⁽¹⁾(u) and S⁽⁰⁾(u); specifically, to develop a test of the null hypothesis H₀: S⁽⁰⁾(u) = S⁽¹⁾(u), u ≥ 0.

We also denote by Z_i the binary treatment assignment for patient i, where Z_i = 1, 0 and make the strong ignorability assumption (Rubin (1978)), or no unmeasured confounders assumption, that Z is conditionally independent of T^(j) given X, denoted by Z ⫫ T^(j)|X. We also make the usual assumption of non-informative censoring; namely, that C ⫫ T^(j)|X, Z. Together these assumptions imply that

$$(Z,C)⫫T^{(j)}\mid X.$$ (1)

We use this assumption in the remainder of this paper and refer to it as assumption (1). Besides assumption (1), we adopt the positivity assumption that 0 < P(Z = 1|X) < 1 and the stable unit treatment value assumption (SUTVA) that one unit’s outcomes are unaffected by another unit’s treatment assignment.

In contrast to the potential outcomes, some of which may not be observed, the data that are observable can be summarized as V_i = (Z_i, X_i, U_i, Δ_i), i = 1, …, N, where, in addition to (Z_i, X_i), already defined, we also observe for individual i their time to death or censoring U_i = min(T_i, C_i) and the failure indicator Δ_i = I(T_i ≤ C_i) where $T_i=Z_i{T}_i^{(1)}+(1-Z_i)T_i^{(0)}$ (the consistency assumption); that is, the time to death on the assigned treatment.

In this section, we develop the log rank type statistic when the covariates collected in the main study are believed to capture all potential confounding. In Bai et al. (2013), we developed a locally-efficient doubly-robust estimator for the treatment-specific survival distributions S^(j)(u) = P(T⁽¹⁾ ≥ u), u ≥ 0, j = 0, 1 as

$$\begin{array}{l}{\widehat{S}}^{(j)}(u)=N^{-1}\sum _{i=1}^N{\varphi }_j^{\ast }(u;V_i)\\ =N^{-1}\sum _{i=1}^N[\frac{{(Z_i)}^j{(1-Z_i)}^{1-j}I(U_i\ge u)}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}{K}_c^{(j)}(u,X_i)}-\frac{(2j-1)\{Z_i-\pi (X_i)\}}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}}{H}^{(j)}(u,X_i)+{\int }_0^u\frac{{(Z_i)}^j{(1-Z_i)}^{1-j}}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}}\frac{\mathrm{d}{M}_c^{(j)}(r,X_i)}{K_c^{(j)}(r,X_i)}\frac{H^{(j)}(u,X_i)}{H^{(j)}(r,X_i)}],\end{array}$$ (2)

where π(X) = P(Z = 1|X) is the propensity score, $K_c^{(j)}(r,X)=P(C\ge r\mid X,Z=j)$ is the conditional survival function of the treatment specific censoring time given X, $\mathrm{d}{M}_c^{(j)}(r,X)$ is the martingale increment for the censoring distribution, namely, $\mathrm{d}{M}_c^{(j)}(r,X)=\mathrm{d}{N}_c(r)-{\lambda }_c^{(j)}(r,X)Y(r)dr$ , N_c(r) = I(U ≤ r, Δ = 0), Y(r) = I(U ≥ r) and ${\lambda }_c^{(j)}(r,X)=\frac{-\mathrm{d}log{K}_c^{(j)}(r,X)}{\mathrm{d}r}$ is the conditional hazard function for C given Z = j and X, and H^(j)(r, X) = P(T^(j) ≥ r|X). Bai et al. (2013) also derived the estimator Ŝ⁽¹⁾(u) − Ŝ⁽⁰⁾(u) and its variance estimator allowing us to compare the survival distributions at a fixed time point u. We note that the monotonicity of the AIPWCC estimator Ŝ^(j)(u) for the survival distribution no longer holds. This is the price paid for increased efficiency. However, this does not affect the theoretical properties of the resulting log rank type test derived in this paper and with sufficiently large sample sizes our simulation studies have shown that the deviation from monotonicity of the AIPWCC estimator is slight.

In a randomized study with data (Z_i, U_i, Δ_i), i = 1, …, N, the log rank test takes the form of

$$\int W_N(u)\{\mathrm{d}{\widehat{\mathit{\Lambda }}}_1(u)-\mathrm{d}{\widehat{\mathit{\Lambda }}}_0(u)\},$$

where $Y_1(u)={\sum }_{i=1}^N{Z}_i{Y}_i(u),Y_0(u)={\sum }_{i=1}^N(1-Z_i)Y_i(u),N_1(u)={\sum }_{i=1}^N{Z}_i{N}_i(u),N_0(u)={\sum }_{i=1}^N(1-Z_i)N_i(u)$ , dΛ̂_j(u) = dN_j(u)/Y_j(u) is the usual treatment specific Nelson-Aalen estimator for the hazard function of the survival distribution for treatment j = 0, 1, and $W_N(u)=N^{-1}\frac{Y_1(u)Y_0(u)}{Y_1(u)+Y_0(u)}$. We note that we normalized W_N(u) so that W_N(u) converges in probability to $w(u)=\frac{P(U\ge u\mid Z=1)\xi \times P(U\ge u\mid Z=0)(1-\xi )}{P(U\ge u\mid Z=1)\xi +P(U\ge u\mid Z=0)(1-\xi )}$, where ξ = P(Z = 1). By analogy, we propose to use ∫ W_N(u){dΛ̂⁽¹⁾(u) − dΛ̂⁽⁰⁾(u)}, where dΛ̂^(j)(u) = −dŜ^(j)(u)/Ŝ^(j)(u), j = 0, 1, to develop a log rank type test of the null hypothesis of treatment equality in observational studies. Here the estimator for the treatment-specific survival distributions Ŝ^(j)(u) uses the doubly robust estimator (2). Define

$$T_N=\int W_N(u)\{\mathrm{d}{\widehat{\mathit{\Lambda }}}^{(1)}(u)-\mathrm{d}{\widehat{\mathit{\Lambda }}}^{(0)}(u)\}=\int W_N(u)\left\{\frac{\mathrm{d}{\widehat{S}}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)}-\frac{\mathrm{d}{\widehat{S}}^{(1)}(u)}{{\widehat{S}}^{(1)}(u)}\right\},$$

and

$$T_N^{\ast }=\int W_N(u)\left[\left\{\frac{\mathrm{d}{\widehat{S}}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)}-\frac{\mathrm{d}{S}^{(0)}(u)}{S^{(0)}(u)}\right\}-\left\{\frac{\mathrm{d}{\widehat{S}}^{(1)}(u)}{{\widehat{S}}^{(1)}(u)}-\frac{\mathrm{d}{S}^{(1)}(u)}{S^{(1)}(u)}\right\}\right],$$

under the null hypothesis of H₀: S⁽¹⁾(u) = S⁽⁰⁾(u), we have $T_N=T_N^{\ast }$, hence $\text{Var}(T_N)=\text{Var}(T_N^{\ast })$. Moreover,

$$\begin{array}{l}{T}_N^{\ast }=\int W_N(u)[\left\{\frac{\mathrm{d}{\widehat{S}}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)}-\frac{\mathrm{d}{S}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)}+\frac{\mathrm{d}{S}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)}-\frac{\mathrm{d}{S}^{(0)}(u)}{S^{(0)}(u)}\right\}\\ -\left\{\frac{\mathrm{d}{\widehat{S}}^{(1)}(u)}{{\widehat{S}}^{(1)}(u)}-\frac{\mathrm{d}{S}^{(1)}(u)}{{\widehat{S}}^{(1)}(u)}+\frac{\mathrm{d}{S}^{(1)}(u)}{{\widehat{S}}^{(1)}(u)}-\frac{\mathrm{d}{S}^{(1)}(u)}{S^{(1)}(u)}\right\}]\\ =\int \left[\frac{W_N(u)}{{\widehat{S}}^{(0)}(u)}\mathrm{d}\left\{{\widehat{S}}^{(0)}(u)-S^{(0)}(u)\right\}-\frac{W_N(u)\mathrm{d}{S}^{(0)}(u)}{{\widehat{S}}^{(0)}(u)S^{(0)}(u)}\left\{{\widehat{S}}^{(0)}(u)-S^{(0)}(u)\right\}\right]\\ -\left[\frac{W_N(u)}{{\widehat{S}}^{(1)}(u)}\mathrm{d}\left\{{\widehat{S}}^{(1)}(u)-S^{(1)}(u)\right\}-\frac{W_N(u)\mathrm{d}{S}^{(1)}(u)}{{\widehat{S}}_1(u)S^{(1)}(u)}\left\{{\widehat{S}}^{(1)}(u)-S^{(1)}(u)\right\}\right].\end{array}$$

Since ${\widehat{S}}^{(j)}(u)-S^{(j)}(u)=\frac{1}{N}{\sum }_{i=1}^N{\varphi }_j(u;V_i)$, where ${\varphi }_j(u;V_i)={\varphi }_j^{\ast }(u;V_i)-S^{(j)}(u)$, then $T_N^{\ast }$ can be approximated by $T_N^{\ast }$ given by

$${\widehat{T}}_N^{\ast }=\frac{1}{N}\sum _{i=1}^N\int \left[\frac{W_N(u)}{{\widehat{S}}^{(0)}(u)}\mathrm{d}{\varphi }_0(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}^{(0)}(u)}{{\{{\widehat{S}}^{(0)}(u)\}}^2}{\varphi }_0(u;V_i)\right]-\left[\frac{W_N(u)}{{\widehat{S}}^{(1)}(u)}\mathrm{d}{\varphi }_1(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}^{(1)}(u)}{{\{{\widehat{S}}^{(1)}(u)\}}^2}{\varphi }_1(u;V_i)\right],$$

in the sense that $N^{1/2}(T_N^{\ast }-{\widehat{T}}_N^{\ast })$ converges in probability to zero. For each subject i, we first calculate

$${\psi }_i=\int \left[\frac{W_N(u)}{{\widehat{S}}^{(0)}(u)}\mathrm{d}{\varphi }_0(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}^{(0)}(u)}{{\{{\widehat{S}}^{(0)}(u)\}}^2}{\varphi }_0(u;V_i)\right]-\left[\frac{W_N(u)}{{\widehat{S}}^{(1)}(u)}\mathrm{d}{\varphi }_1(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}^{(1)}(u)}{{\{{\widehat{S}}^{(1)}(u)\}}^2}{\varphi }_1(u;V_i)\right],i=1,\dots ,N.$$

Then we compute the empirical variance of ψ_i,

$$\text{Var}({\psi }_i)=\frac{1}{N-1}\sum _{i=1}^N{\left({\psi }_i-\frac{1}{N}\sum _{j=1}^N{\psi }_j\right)}^2.$$

Finally, we use Var(ψ_i)/N as the estimator to $\widehat{\text{Var}}({\widehat{T}}_N^{\ast })$. The log rank type statistic takes the form

$$G=\frac{T_N}{\sqrt{\widehat{\text{Var}}({\widehat{T}}_N^{\ast })}},$$ (3)

which asymptotically follows a standard normal distribution N(0, 1) under the null hypothesis of H₀: S⁽¹⁾(u) = S⁽⁰⁾(u). Hence, the null hypothesis is rejected at α level when |G| > Z_α/2, where Z_α/2 is 1 − α/2 quantile of standard normal distribution.

In Bai et al. (2013), it is shown that Ŝ^(j)(u), j = 0, 1, is doubly-robust; that is, if either the estimator for the coarsening defined through the estimator for P(Z = 1|X) = π(X) and the estimator for $P(C\ge r\mid Z=j,X)=K_c^{(j)}(r,X)$ are both consistent or the estimator for P(T^(j) ≥ r|X) = H^(j)(r, X) is consistent, then (2) will be a consistent estimator for S^(j)(u). Hence, the numerator T_N is also a doubly-robust estimator of ∫ w(u){dΛ̂₁(u) − dΛ̂₀(u)}; hence has expectation zero under the null hypothesis if either π(X) and $K_c^{(j)}(r,X)$ are both consistently estimated or H^(j)(r, X) is consistently estimated. The denominator estimator $\sqrt{\widehat{\text{Var}}({\widehat{T}}_N^{\ast })}$, however, is not guaranteed to be valid under model misspecification. If the propensity score π(X) and the conditional censoring distribution $K_c^{(j)}(r,X)$ were consistently estimated whereas, the conditional survival distribution H^(j)(r, X) was not, then the variance estimator would be conservative (Hubbard et al. (2000); Tsiatis (2006) Theorem 9.1). The resulting test statistic $\frac{T_N}{\sqrt{\widehat{\text{Var}}({\widehat{T}}_N^{\ast })}}$ would follow a normal distribution with mean 0 and variance less than 1. On the other hand, when the estimator for H^(j)(r, X) is consistent and either of the estimators for π(X) or $K_c^{(j)}(r,X)$ are not, then the resulting denominator $\sqrt{\widehat{\text{Var}}({\widehat{T}}_N^{\ast })}$ may be biased and there is no theoretical result on the direction or extent of the bias. In such cases, it is suggested to use a bootstrap estimator of the asymptotic variance. To be more specific, we resample the dataset (Z_i, X_i, U_i, Δ_i), i = 1, …, n, with replacement B times. In each bootstrap replicate, we compute the numerator $T_N^{(b)}$, b = 1, …, B. We then compute the sample standard deviation of $T_N^{(b)}$; b = 1, …, B, and denote that by $\sqrt{\text{Var}\{T_N^{(b)}\}}$. We propose the bootstrap test statistics as $G_{\mathit{bootstrap}}=\frac{T_N}{\sqrt{\text{Var}\{T_N^{(b)}\}}}$. As shown later in our simulation studies in Section 4, the bootstrap version of these test statistics have good asymptotic properties under varying model misspecification.

Similar to the ordinary log rank test, under local alternatives, these test statistics follow a normal distribution with a non-centrality parameter and variance 1. We compare the performance of this statistic with other competing methods in terms of power under various alternatives in the simulation studies in Section 4.

One thing worth pointing out is that both the numerator and denominator of (3) involve the reciprocal of Ŝ^(j)(u), j = 0, 1, which tends to be unstable at the tail of survival curve. Therefore, in order to lessen this problem one may truncate the range of integration of our test statistic. The validity of the distribution of this truncated test statistic will still hold under the null hypothesis. However, with truncation there is potential loss of efficiency by not using all the survival data. Thus, we need to evaluate the balance between increased stability versus loss of efficiency in the choice of the truncation point. This issue will also be examined in our simulation studies.

3 Test Statistics in Stratified Sampling with Additional Information from a Subsample

As discussed in Bai et al. (2013), the inference derived in Section 2 will be valid only when the critical no unmeasured confounder assumption Z ⫫ T^(j)|X holds for covariates collected in the main study. If this assumption is questionable then one may decide to collect additional covariate information in a substudy in order to make this assumption tenable. This indeed was the case for the ASCERT study where additional covariates were collected using a stratified sampling design. We now show how one can construct a valid log rank type test for comparing treatment-specific survival distributions where a substudy using a stratified sampling design to collect such additional covariates was used.

Using the same notation as in Bai et al. (2013), X₁ denotes the covariates collected in the main study and X₂ denotes the additional covariates collected on the subsample. Use F_i = (U_i, Δ_i, Z_i, X_1i, X_2i), i = 1, …, N to denote the “full data” and

$$\begin{array}{l}{\varphi }_j^{\ast }(u;F_i)=[\frac{{(Z_i)}^j{(1-Z_i)}^{1-j}I(U_i\ge u)}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}{K}_c^{(j)}(u,X_i)}\\ -\frac{(2j-1)\{Z_i-\pi (X_i)\}}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}}{H}^{(j)}(u,X_i)\\ +{\int }_0^u\frac{{(Z_i)}^j{(1-Z_i)}^{1-j}}{{\{\pi (X_i)\}}^j{\{1-\pi (X_i)\}}^{1-j}}\frac{\mathrm{d}{M}_c^{(j)}(r,X_i)}{K_c^{(j)}(r,X_i)}\frac{H^{(j)}(u,X_i)}{H^{(j)}(r,X_i)}]\end{array}$$

to denote the i-th element of estimating equation that would have been used to estimate S^(j)(u) if we had access to all the full data in the main study. This is similar to the estimating equation (2) but using both X₁ and X₂. As in the ASCERT study, a subsample may be obtained using a stratified sampling design in which K strata are identified based on (Z, X₁). Let ε denote the stratum indicator taking values 1, …, K such that $N_k={\sum }_{i=1}^{N}I({\epsilon }_i=k)$ is the number of subjects in stratum k, k = 1, …, K, then from stratum k, a fixed (by design) number of subjects n_k are sampled from the N_k subjects at random without replacement. As shown in Bai et al. (2013), the doubly-robust estimator for the treatment-specific survival probability at time point u is:

$${\widehat{S}}_{\mathit{start}}^{(j)}(u)=\frac{1}{N}\sum _{i=1}^N\left[\frac{R_i}{\eta ({\epsilon }_i)}{\widehat{\varphi }}_j^{\ast }(u;F_i)-\left\{\frac{R_i-\eta ({\epsilon }_i)}{\eta ({\epsilon }_i)}\right\}{h}^{(j)}(u;V_i)\right],$$ (4)

and its variance can be estimated by the summation of

$$N^{-1}\left[\sum _{k=1}^K{p}_k{\mu }_k^2({\varphi }_j)-{\left\{\sum _{k=1}^K{p}_k{\mu }_k({\varphi }_j)\right\}}^2\right]$$ (5)

and

$$\sum _{k=1}^K{\left(\frac{N_k}{N}\right)}^2\left[\left\{{\text{Var}}_k({\varphi }_j)-2{\text{Cov}}_k({\varphi }_j,h^{(j)})+{\text{Var}}_k(h^{(j)})\right\}\left(\frac{1}{n_k}-\frac{1}{N_k}\right)+{\text{Var}}_k({\varphi }_j)\left(\frac{1}{N_k}\right)\right],$$ (6)

where R_i is the subsample indicator, η(ε_i) = P(R_i = 1|ε_i), $h^{(j)}(u;V_i)=E\{{\varphi }_j^{\ast }(F_i)\mid V_i,{\epsilon }_i\}$, p_k = P(ε = k), ${\varphi }_j={\varphi }_j^{\ast }-S^{(j)}(u)$, μ_k(ϕ_j) = E[ϕ_j{F, S^(j)(u)}|ε = k], μ_k(h^(j)) = E[h^(j)(V)|ε = k], Var_k(ϕ_j) = Var[ϕ_j{F, S^(j)(u)}|ε = k], Var_k(h^(j)) = Var{h^(j)(V)|ε = k}, and Cov_k(ϕ_j, h^(j)) = Cov[ϕ_j{F, S^(j)(u)}, h^(j)(V)|ε = k]. Note that $h^{(j)}(u;V_i)=E\{{\varphi }_j^{\ast }(F_i)\mid V_i,{\epsilon }_i\}$, also referred to as the optimal h function or h^opt, is chosen such that the variance of Ŝ^(j)(u) is minimized. Such efficiency is achieved by recovering information from observations in the main study but not selected in the subsample. A simple and effective estimating strategy of h^opt is proposed in Bai et al. (2013), suggesting that we fit a simple linear regression of ${\varphi }_j^{\ast }(F)$ on ${\varphi }_j^{\ast }(V)$ by strata. We adopt this estimation method in the simulation and data analysis section later.

As in the case of the main study, the numerator of the test statistic is given by

$$T_{N,\mathit{strat}}=\int W_N(u)\left\{\frac{\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}-\frac{\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\right\}.$$

Under the null hypothesis, we have

$$\begin{array}{l}{T}_{N,\mathit{strat}}=T_{N,\mathit{strat}}^{\ast }\\ =\int \left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}\mathrm{d}\left\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)-S^{(0)}(u)\right\}-\frac{W_N(u)\mathrm{d}{S}^{(0)}(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)S^{(0)}(u)}\left\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)-S^{(0)}(u)\right\}\right]-\left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\mathrm{d}\left\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)-S^{(1)}(u)\right\}-\frac{W_N(u)\mathrm{d}{S}^{(1)}(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)S^{(1)}(u)}\left\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)-S^{(1)}(u)\right\}\right].\end{array}$$

Hence,

$$\begin{array}{l}{\widehat{T}}_{N,\mathit{strat}}^{\ast }=\frac{1}{N}\sum _{i=1}^N(\frac{R_i}{\eta ({\epsilon }_i)}\int \left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}\mathrm{d}{\varphi }_0(u;F_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)\}}^2}{\varphi }_0(u;F_i)\right]\\ -\left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\mathrm{d}{\varphi }_1(u;F_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)\}}^2}{\varphi }_1(u;F_i)\right]\\ -\frac{R_i-\eta ({\epsilon }_i)}{\eta ({\epsilon }_i)}\int \left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}\mathrm{d}{h}^{(0)}(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)\}}^2}{h}^{(0)}(u;V_i)\right]\\ -\left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\mathrm{d}{h}^{(1)}(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)\}}^2}h(1)(u;V_i)\right]).\end{array}$$

Notice that ${\widehat{T}}_{N,\mathit{strat}}^{\ast }$ is of the same form as the AIPWCC stratified estimator (4), its variance can also be estimated by adding up the estimator of the variance of the conditional expectation terms like (5) and the estimator of the expectation of the conditional variance term like (6). To be specific, for each subject i, we first calculate

$${\psi }_{i,\mathit{strat}}=\frac{R_i}{\eta ({\epsilon }_i)}{\zeta }_i^{\ast }-\frac{R_i-\eta ({\epsilon }_i)}{\eta ({\epsilon }_i)}{\gamma }_i,$$

where

$${\zeta }_i^{\ast }=\int \left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}\mathrm{d}{\varphi }_0(u;F_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)\}}^2}{\varphi }_0(u;F_i)\right]-\left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\mathrm{d}{\varphi }_1(u;F_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)\}}^2}{\varphi }_1(u;F_i)\right],$$

and

$${\gamma }_i=\int \left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}\mathrm{d}{h}^{(0)}(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(0)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(0)}(u)\}}^2}{h}^{(0)}(u;V_i)\right]-\left[\frac{W_N(u)}{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}\mathrm{d}{h}^{(1)}(u;V_i)-\frac{W_N(u)\mathrm{d}{\widehat{S}}_{\mathit{strat}}^{(1)}(u)}{{\{{\widehat{S}}_{\mathit{strat}}^{(1)}(u)\}}^2}{h}^{(1)}(u;V_i)\right].$$

Then the variance $\widehat{\text{Var}}({\widehat{T}}_{N,\mathit{strat}}^{\ast })$ can be estimated by the summation of

$$N^{-1}\left[\sum _{k=1}^K{p}_k{\mu }_k^2(\zeta )-{\left\{\sum _{k=1}^K{p}_k{\mu }_k(\zeta )\right\}}^2\right]$$

and

$$\sum _{k=1}^K{\left(\frac{N_k}{N}\right)}^2\left[\{{\text{Var}}_k(\zeta )-2{\text{Cov}}_k(\zeta ,\gamma )+{\text{Var}}_k(\gamma )\}\left(\frac{1}{n_k}-\frac{1}{N_k}\right)+{\text{Var}}_k(\zeta )\left(\frac{1}{N_k}\right)\right],$$

where ${\zeta }_i={\zeta }_i^{\ast }-{\widehat{T}}_{N,\mathit{strat}}^{\ast }$, μ_k(ζ) = Σ_{{i:εi = k, Ri = 1}} ζ_i/n_k, μ_k(γ) = Σ_{{i:εi = k}}γ_i/N_k, Var_k(ζ) = Var_{{i:εi = k, Ri = 1}}(ζ_i), Var_k(γ) = Var_{{i:εi = k}}(γ_i) and Cov_k(ζ, γ) = Cov_{{i:εi = k, Ri = 1}}(ζ_i, γ_i). The final log rank type statistic in the stratified sampling framework is given by

$$G_{\mathit{strat}}=\frac{T_{N,\mathit{strat}}}{\sqrt{\widehat{\text{Var}}({\widehat{T}}_{N,\mathit{strat}}^{\ast })}}.$$ (7)

The discussion at the end of Section 2 also applies to the case with stratified sampling: if all models are correctly specified, the test statistics follow a standard normal distribution under the null hypothesis and follow a noncentral normal distribution with variance 1 under the alternative hypothesis. Misspecification of the regression model H^(j)(r, X) would result in a test statistic with variance less than 1; misspecification of the propensity model π(X) and/or $K_c^{(j)}(r,X)$ would affect the variance of the test statistic in an unknown direction. Similar to the case of the main study, the nonparametric bootstrap procedure might provide a more accurate estimator of the asymptotic variance. One way to conduct the bootstrap sampling is to randomly sample with replacement the main study subjects and substudy subjects separately. That is, given a bootstrap number B, for each bootstrap replicate b = 1, …, B, a total of ${\sum }_{i=1}^N{R}_i$ subjects are sampled with replacement among all subject with R_i = 1 and a total of ${\sum }_{i=1}^N(1-R_i)$ subjects are sampled with replacement among all subject with R_i = 0. Combining the new N subjects sampled from the b-th bootstrap replicate, we compute $T_{N,\mathit{strat}}^{(b)}$ . The bootstrap test statistics in stratified sampling take would take the form $G_{\mathit{strat},\mathit{bootstrap}}=\frac{T_{N,\mathit{strat}}}{\sqrt{\text{Var}\{T_{N,\mathit{strat}}^{(b)}\}}}$, where $\sqrt{\text{Var}\{T_{N,\mathit{strat}}^{(b)}\}}$ is the sample standard deviation of $T_{N,\mathit{strat}}^{(b)}$, b = 1, …, B. As shown in the simulation study, the bootstrap version of test performs better compared to (7) in the case of misspecification. Also, appropriate truncation could result in more stable performance with possible slight efficiency loss.

4 Simulations

4.1 Simulation 1: Main Study Only

In this section, we conduct a simulation study on the performance of test statistics (3) under both the null hypothesis and various alternative hypotheses. Different type of misspecification scenarios, including misspecification of propensity models and of regression models, are also considered.

In this simulation, we generated 500 replications and within each replicate, N = 2, 000 observations are generated as follows: B ~ Bernoulli(0.5), W ~ N(0, 1) and X₂ ~ N (0, 1), mutually independent. We denote by X₁ = (B, W, W²)T, and X = ( $X={(X_1^T,X_2)}^T$, X₂)^T. The treatment assignment propensity is generated by $P(Z=1\mid X)=\frac{exp(0.1B+0.1W+0.5W^2+0.5X_2)}{1+exp(0.1B+0.1W+0.5W^2+0.5X_2)}$, and the treatment-specific hazard functions are given by λ_Z=1(t|X) = λ_Z=0(t|X) = exp(−0.5 + 0.1B + 0.1W + 0.5W² + 0.5X₂). We also generated an independent censoring variable C as Uniform(0, 4). The censoring rate is approximately 28%. This is a scenario under the null hypothesis and we expect the test statistics (3) to follow a standard normal distribution.

With all models, π(X), $K_c^{(j)}(r,X)$ and H^(j)(r, X), being correctly specified, the performance of the log rank type test statistics (3) is summarized in Table 1. It is expected that the test statistics follow the standard normal distribution and the type I error rate is indeed close to 0.05, with or without truncation. We also computed the ordinary log rank test and because of confounding, this test does not provide valid inference and rejects the null hypothesis with probability 1. The mean of ordinary log rank test is 7.05 and the standard deviation is 0.99.

Table 1.

Simulation 1, under the null hypothesis, the performance of test statistics (3) when there is no stratified sampling. Sample size N=2000. Correct model specification.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ 1.96)	Pr(|stat| ≥ 1.96)
no	0.028	0.026	0.054
at 3.50	0.026	0.026	0.052
at 3.25	0.026	0.026	0.052
at 3.00	0.026	0.026	0.052
at 2.75	0.026	0.028	0.054
at 2.50	0.028	0.030	0.058

If we misspecify the propensity model by leaving out the W² term in π(X) and keeping the other two models $K_c^{(j)}(r,X)$ and H^(j)(r, X) correctly specified, the performance of log rank type test statistics (3) is summarized in Table 2. Theoretically, in this scenario, the test statistics should still follow the normal distribution with mean 0 but not necessarily a variance of one. In this scenario the variance estimator was close to one and hence the nominal level of significance was close to 0.05, however, this is not guaranteed, at least theoretically, to occur in general. Table 3 presents the bootstrap version of test statistics G_bootstrap with B = 100 bootstrap resampling under wrong propensity model. We see after appropriate truncation, the bootstrap test provides nominal level of significance close to 0.05.

Table 2.

Simulation 1, under the null hypothesis, the performance of test statistics (3) when there is no stratified sampling. Sample size N=2000. Correct regression models and wrong propensity models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ −1.96)	Pr(|stat| ≥ 1.96)
no	0.026	0.024	0.050
at 3.50	0.028	0.022	0.050
at 3.25	0.030	0.020	0.050
at 3.00	0.028	0.022	0.050
at 2.75	0.028	0.018	0.046
at 2.50	0.028	0.022	0.050

Table 3.

Simulation 1, under the null hypothesis, the performance of bootstrap test statistics G_bootstrap when there is no stratified sampling. Sample size N=2000. Correct regression models and wrong propensity models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ −1.96)	Pr(|stat| ≥ 1.96)
no	0.030	0.016	0.046
at 3.50	0.024	0.032	0.056
at 3.25	0.024	0.034	0.058
at 3.00	0.026	0.032	0.058
at 2.75	0.026	0.034	0.060
at 2.50	0.028	0.034	0.062

We also considered misspecification of the regression models by leaving out the W² term in H^(j)(r, X) and keeping the propensity models π(X) and $K_c^{(j)}(r,X)$ correctly specified, the performance of log rank type test statistics (3) is summarized in Table 4. Theoretically, in this scenario, the test statistics should still follow the normal distribution with mean 0 and variance less than 1, which is indeed supported by our simulation results. The bootstrap version of test statistics G_bootstrap with B = 100 bootstrap resampling under wrong regression model is presented in Table 5. By using the bootstrap variance, the resulting log rank test statistic G_bootstrap provides nominal significance level close to 0.05, even when the regression model is misspecified.

Table 4.

Simulation 1, under the null hypothesis, the performance of test statistics (3) when there is no stratified sampling. Sample size N=2000. Correct propensity models and wrong regression models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ −1.96)	Pr(|stat| ≥ 1.96)
no	0.016	0.010	0.026
at 3.50	0.020	0.010	0.030
at 3.25	0.020	0.010	0.030
at 3.00	0.020	0.010	0.030
at 2.75	0.020	0.008	0.028
at 2.50	0.014	0.008	0.022

Table 5.

Simulation 1, under the null hypothesis, the performance of Bootstrap test statistics G_boostrap when there is no stratified sampling. Sample size N=2000. Correct propensity models and wrong regression models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ − 1.96)	Pr(|stat| ≥ 1.96)
no	0.028	0.020	0.048
at 3.50	0.032	0.018	0.050
at 3.25	0.032	0.016	0.048
at 3.00	0.034	0.022	0.056
at 2.75	0.032	0.022	0.054
at 2.50	0.036	0.018	0.054

Under the alternative hypothesis of ${\lambda }_{Z=1}(t\mid X)=exp(-{\beta }_0^{(1)}+0.1B+0.1W+0.5W^2+0.5X_2)$, and λ_Z=0(t|X) = exp(−0.5+0.1B+0.1W+0.5W²+ 0.5X₂), the power performance of the test statistics (3) is summarized in Table 6. The censoring rates are approximately 29% for ${\beta }_0^{(1)}=0.6$ and 30% for ${\beta }_0^{(1)}=0.7$. The model used to generate the data was a proportional hazards regression model with covariates X and Z, and here the null hypothesis would correspond to ${\beta }_0^{(1)}=0.5$. Consequently, for this model the most powerful test of the null hypothesis versus alternatives ${\beta }_0^{(1)}\ne 0.5$ is obtained by fitting a Cox proportional hazard model with covariate X and Z to the data and using the estimated coefficient of Z divided by its estimated standard deviation as the gold standard test statistic. The power of this gold standard test statistic is 0.376 when ${\beta }_0^{(1)}=0.6$ and 0.906 when ${\beta }_0^{(1)}=0.7$. This indicates that (3) is comparable to the optimal model based gold standard test.

Table 6.

Simulation 1, under the alternative hypothesis, the power of test statistics (3) and gold standard statistic (GS) when there is no stratified sampling. Sample size N=2000. Correct model specification.

truncation	Power
	${\beta }_0^{(1)}=0.6$		${\beta }_0^{(1)}=0.7$
	AIPWCC	GS	AIPWCC	GS
no	0.346	0.376	0.884	0.906
at 3.50	0.352	0.374	0.898	0.908
at 3.25	0.350	0.376	0.896	0.902
at 3.00	0.352	0.372	0.896	0.908
at 2.75	0.344	0.368	0.886	0.906
at 2.50	0.350	0.370	0.890	0.908

4.2 Simulation 2: With Substudy

In this section, we consider the performance of test statistics (7) under both null and alternative hypotheses. Similar to the previous study, we also compute the test under different misspecification scenarios, including misspecification of the propensity models and regression models.

The data generating mechanism is the same as the previous studies, where X₁ = (B, W, W²)^T is collected on N = 2000 observations in the main study and X₂ is only observable for a random subsample of size n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300, where n_jk denotes the number of subjects in stratum Z = j and B = k − 1, for j = 0, 1, k = 1, 2. For this scenario we also computed the test statistics (3) using only X₁ and not adjusting for the residual confounding of X₂. As expected, the resulting test statistics were biased leading to a rejection rate of roughly 0.95 under the null hypothesis for a test at the nominal 0.05 level.

In order to get consistent statistics, we need to adjust for additional con-founding captured by the substudy, X₂. Besides (7), which will be referred to as AIPWCC stratified test statistics, we compute similar test statistics with no augmentation terms. That is, we compute (7) by substituting h^(j)(u; V) = 0, j = 0, 1 in both the numerator and denominator. This estimator, referred to as weighted test statistics, $G_{\mathit{strat}}^{\mathit{weighted}}$, only include the subjects being selected into the substudy and we use information collected on them from both the main study and the substudy. Hence, it is expected to be less efficient than the AIPWCC stratified test statistics. Similar to G_{strat,bootstrap}, a bootstrap version of weighted test statistics, denoted by $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$, could be computed by using the sample standard deviation as the denominator. The performance of G_{strat,bootstrap} and $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$ will be compared in the case of model misspecification.

When all models, π(X), $K_c^{(j)}(r,X)$ and H^(j)(r, X), are correctly specified, the performance of AIPWCC stratified test statistics and weighted test statistics are summarized in Table 7. Both test statistics follows the standard normal distribution and the type I error rate is close to the nominal 0.05 level, regardless of the degree of truncation. Similar to the case of main study only, the ordinary log rank test statistic does not account for the confounding appropriately and leads to biased result with mean 7.00 and standard deviation 1.01.

Table 7.

Simulation 2, under the null hypothesis, the performance of AIPWCC stratified test statistics and weighted test statistics with stratified sampling. In each cell, the left column corresponds to AIPWCC stratified test statistics and the right column corresponds to weighted test statistics. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct model specification.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ 1.96)	Pr(|stat| ≥ 1.96)
no	0.028—0.026	0.024—0.028	0.052—0.054
at 3.50	0.028—0.026	0.026—0.032	0.054—0.058
at 3.25	0.022—0.026	0.030—0.032	0.052—0.058
at 3.00	0.024—0.022	0.030—0.032	0.054—0.054
at 2.75	0.026—0.020	0.028—0.032	0.054—0.052
at 2.50	0.026—0.022	0.026—0.032	0.052—0.054

When the propensity model is misspecified by leaving out the W² term in π(X) and the other two models $K_c^{(j)}(r,X)$ and H^(j)(r, X) are correctly specified, the performance of the AIPWCC stratified test statistics and weighted test statistics are summarized in Table 8. Theoretically, in this scenario, the test statistics should still follow a normal distribution with mean 0 but with variance that may not equal one; however, our simulation results show that the variance of both test statistics are close to 1 in this setting leading to nominal significance levels close to 0.05. Table 9 presents the result of the bootstrap test statistics G_{strat,bootstrap} and $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$. Both provide valid inference with nominal significance level.

Table 8.

Simulation 2, under the null hypothesis, the performance of AIPWCC stratified test statistics and weighted test statistics with stratified sampling. In each cell, the left column corresponds to AIPWCC stratified test statistics and the right column corresponds to weighted test statistics. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct regression models and wrong propensity models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ − 1.96)	Pr(|stat| ≥ 1.96)
no	0.018—0.010	0.024—0.030	0.042—0.040
at 3.50	0.026—0.012	0.026—0.032	0.052—0.044
at 3.25	0.024—0.008	0.032—0.034	0.056—0.042
at 3.00	0.026—0.006	0.024—0.032	0.050—0.038
at 2.75	0.026—0.006	0.024—0.032	0.050—0.038
at 2.50	0.030—0.008	0.026—0.032	0.056—0.040

Table 9.

Simulation 2, under the null hypothesis, the performance of AIPWCC bootstrap stratified test statistics G_{strat,bootstrap} and bootstrap weighted test statistics $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$ with stratified sampling. In each cell, the left column corresponds to G_{strat,bootstrap} and the right column corresponds to $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct regression models and wrong propensity models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ − 1.96)	Pr(|stat| ≥ 1.96)
no	0.018—0.010	0.016—0.034	0.034—0.044
at 3.50	0.028—0.014	0.034—0.034	0.062—0.048
at 3.25	0.032—0.010	0.038—0.032	0.070—0.042
at 3.00	0.028—0.008	0.032—0.038	0.060—0.046
at 2.75	0.026—0.012	0.030—0.036	0.056—0.048
at 2.50	0.034—0.010	0.032—0.030	0.066—0.040

If we misspecify the regression models by leaving out the W² term in H^(j)(r, X) and keep the propensity models π(X) and $K_c^{(j)}(r,X)$ correctly specified, the performance of the AIPWCC stratified test statistics and weighted test statistics are summarized in Table 10. In this scenario, both test statistics should still follow the normal distribution with mean 0 and variance less than 1. In particular, the variance of the AIPWCC stratified test statistics are larger than the weighted test statistics. This indeed is what we would expect. The bootstrap test statistics G_{strat,bootstrap} and $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$ are summarized in Table 11. Compared with Table 10, we see the bootstrap version of tests are more accurate. Hence, it would be suggested to perform the bootstrap test when there exists the possibility of misspecification.

Table 10.

Simulation 2, under the null hypothesis, the performance of AIPWCC stratified test statistics and weighted test statistics with stratified sampling. In each cell, the left column corresponds to AIPWCC stratified test statistics and the right column corresponds to weighted test statistics. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct propensity models and wrong regression models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ − 1.96)	Pr(|stat| ≥ 1.96)
no	0.018—0.018	0.014—0.008	0.032—0.026
at 3.50	0.022—0.014	0.012—0.010	0.034—0.024
at 3.25	0.018—0.014	0.012—0.012	0.030—0.026
at 3.00	0.018—0.012	0.010—0.012	0.028—0.024
at 2.75	0.016—0.010	0.010—0.014	0.026—0.024
at 2.50	0.018—0.010	0.014—0.016	0.032—0.026

Table 11.

Simulation 2, under the null hypothesis, the performance of AIPWCC bootstrap stratified test statistics G_{strat,bootstrap} and bootstrap weighted test statistics $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$ with stratified sampling. In each cell, the left column corresponds to G_{strat,bootstrap} and the right column corresponds to $G_{\mathit{strat},\mathit{bootstrap}}^{\mathit{weighted}}$. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct propensity models and wrong regression models.

truncation	Pr(stat ≥ 1.96)	Pr(stat ≤ −1.96)	Pr(|stat| ≥ 1.96)
no	0.022—0.020	0.024—0.014	0.046—0.034
at 3.50	0.022—0.018	0.028—0.022	0.050—0.040
at 3.25	0.022—0.018	0.028—0.020	0.050—0.038
at 3.00	0.020—0.014	0.024—0.020	0.044—0.034
at 2.75	0.020—0.014	0.026—0.024	0.046—0.038
at 2.50	0.020—0.018	0.028—0.022	0.048—0.040

Under the alternative hypothesis of ${\lambda }_{Z=1}(t\mid X)=exp(-{\beta }_0^{(1)}+0.1B+0.1W+0.5W^2+0.5X_2)$, and λ_Z=0(t|X) = exp(−0.5+0.1B+0.1W+0.5W²+0.5X₂), the power performance of the test statistics (7) is summarized in Table 12. As expected, the AIPWCC stratified test statistics are more powerful than weighted test statistics. Again, we computed the gold standard test by fitting a proportional hazard Cox model with covariate X and treatment assignment Z on the subsample observations. We use the coefficient of Z divided by its estimated standard deviation as the gold standard test statistic. Note that for those subject not selected in the subsample, their information will not be used to compute the gold standard test statistics. The simulation results indicate that the AIPWCC stratified test statistics are more powerful than the gold standard test. The efficiency gain comes from the use of the main study observations that are not selected in the subsample.

Table 12.

Simulation 2, under the alternative hypothesis, the power of AIPWCC stratified test statistics (AIPWCC), weighted test statistics (weighted) and gold standard (GS) statitics with stratified sampling. Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct models specification.

truncation	Power
	${\beta }_0^{(1)}=0.6$			${\beta }_0^{(1)}=0.7$
	AIPWCC	weighted	GS	AIPWCC	weighted	GS
no	0.300	0.210	0.268	0.736	0.694	0.760
at 3.50	0.342	0.226	0.272	0.856	0.706	0.748
at 3.25	0.350	0.236	0.268	0.860	0.714	0.752
at 3.00	0.362	0.236	0.270	0.866	0.712	0.758
at 2.75	0.348	0.226	0.258	0.854	0.706	0.748
at 2.50	0.340	0.234	0.270	0.850	0.714	0.748

To further illustrate the efficiency gain of the AIPWCC stratified test statistics we also considered a simulation scenario where X₂ is not a confounding variable; that is, the coefficients associated with X₂ in both the regression model and propensity score model were set to zero. Together with the stratified test statistics we also computed the AIPWCC test statistics (3) using only the data from the main study, (AIPWCC (main)) which is a valid test under this scenario. The results are summarized in Table 13. If in fact the additional covariates are not necessary to adjust for confounding, the AIPWCC stratified test statistics are still able to recover much of the information from the main study subjects resulting in comparable power.

Table 13.

Simulation 2, under the alternative hypothesis, the power comparison of AIPWCC stratified test statistics (AIPWCC), weighted test statistics (weighted), gold standard (GS) statitics with stratified sampling, and main study AIPWCC test statistics (3) (AIPWCC (main)). Sample size N=2000, n₁₁ = n₁₂ = n₀₁ = n₀₂ = 300. Correct models specification.

truncation	${\beta }_0^{(1)}=0.7$
	AIPWCC	weighted	GS	AIPWCC (main)
no	0.810	0.766	0.808	0.942
at 3.50	0.934	0.774	0.810	0.938
at 3.25	0.932	0.772	0.806	0.940
at 3.00	0.926	0.770	0.800	0.934
at 2.75	0.926	0.772	0.798	0.928
at 2.50	0.922	0.764	0.792	0.922

We noticed in our simulation studies that the greatest power was obtained with modest truncation; in our case, truncation at 3.5. The value 3.5 corresponds roughly to the 90th percentile of the observed failure times. Thus truncating at the 90th percentile of the observed failure times seems to be a good compromise to balance the instability of the test statistic in the tail of the distribution with the loss of efficiency in not using all the data.

5 Analysis on ASCERT Data

In this section, we apply the proposed log rank test statistics to data from the ASCERT study. The goal is to test the null hypothesis that there is no significant difference between PCI and CABG using baseline covariate data from the main ASCERT study database augmented with information on coronary anatomy from a subsample of patients in the ASCERT angiographic companion study. The main study consisted of records from 9, 800 patients in the ASCERT database who underwent a coronary revascularization procedure (CABG or PCI) at one of 54 hospitals participating in the ASCERT study that agreed to be part of the companion study. Subsequently it was determined that some patients were ineligible for analysis. Consequently, the main study participants consisted of 7, 393 eligible patients among the 9, 800 patients in the 54 hospitals. For the purpose of this analysis we considered the patients from the 54 hospitals of the companion study to be the main focus of inference. Twenty-eight covariates were used on the full sample which included demographics (e.g., age, sex), risk factors (e.g., body mass index, smoking), symptoms and history of cardiovascular disease (e.g., chest pain, congestive heart failure), and comorbidities (e.g., diabetes). The subsample includes records from approximately 2, 000 patients chosen by design (roughly 500 in each of the four strata determined by all combinations of the two treatments and whether or not the patients had two- or three-vessel disease). After consideration of eligibility issue, 1, 554 eligible patients remained in the sub- study for analysis. Information collected on the subsample includes features of the patient’s coronary anatomy (e.g., left-side dominance) and features of each individual blockage (e.g., lesion length, tortuosity, calcification, degree of stenosis).

We are interested in testing the treatment effect between PCI and CABG up to time point 1 month and 4 years. Figure 1 displays the estimated treatment-specific curves using the AIPWCC estimator on the main study (left panel), the stratified AIPWCC estimator with h = 0 (middle panel) and with h = h^opt (right panel), respectively. The resulting survival curves are very similar: PCI has better short-term survival probability while CABG has better long-term survival probability. This result suggests that the confounding of additional covariates collected in the substudy is not very influential.

Fig. 1.

Estimated AIPWCC and AIPWCC Stratified Survival Estimator for CABG and PCI.

Table 14 shows the log rank test statistics on the main study, weighted test statistics, AIPWCC stratified test statistics and ordinary log rank test statistics truncated at 1 month and 4 years. Consistent with the treatment-specific curves, the log rank test statistics in the main study suggests PCI is significantly better than CABG up to 1 month and CABG is better over 4 years. The weighted test statistics only use data from subsample and hence lack power. Meanwhile, the AIPWCC stratified test statistics are able to recover more information by making use of subjects in the main study. For completeness we also computed the ordinary log rank test. In this example the ordinary logrank test exaggerates the treatment difference due to the fact that it does not adjust for confounding (neither in main study nor substudy).

Table 14.

Log rank tests applied on the ASCERT data, including log rank test statistics on main study, weighted test statistics, AIPWCC stratified test statistics and ordinary log-rank test statistics. All tests are truncated at time point of interest: 1 month and 4 years.

Truncation	Main Study	Weighted	AIPWCC Stratified	Ordinary
1 month	2.87	0.39	2.47	2.31
4 years	−2.34	−0.66	−1.91	−3.97

6 Discussion

In this paper, we use semiparametric theory to derive log rank type statistics in observational survival studies. When the important confounding covariates are captured in the main study, the resulting test statistic is doubly-robust. When the issue of confounding is still in question within the main study, a substudy could be conducted to collect additional potential confounding variables. We proposed AIPWCC stratified test statistics in such a scenario, which maintains the double-robustness property.