Omnibus test for restricted mean survival time based on influence function

Abstract

The restricted mean survival time (RMST), which evaluates the expected survival time up to a pre-specified time point $\tau$ , has been widely used to summarize the survival distribution due to its robustness and straightforward interpretation. In comparative studies with time-to-event data, the RMST-based test has been utilized as an alternative to the classic log-rank test because the power of the log-rank test deteriorates when the proportional hazards assumption is violated. To overcome the challenge of selecting an appropriate time point $\tau$ , we develop an RMST-based omnibus Wald test to detect the survival difference between two groups throughout the study follow-up period. Treating a vector of RMSTs at multiple quantile-based time points as a statistical functional, we construct a Wald ${\chi }^2$ test statistic and derive its asymptotic distribution using the influence function. We further propose a new procedure based on the influence function to estimate the asymptotic covariance matrix in contrast to the usual bootstrap method. Simulations under different scenarios validate the size of our RMST-based omnibus test and demonstrate its advantage over the existing tests in power, especially when the true survival functions cross within the study follow-up period. For illustration, the proposed test is applied to two real datasets, which demonstrate its power and applicability in various situations.

1. Introduction

In statistical analysis of clinical studies with time-to-event data, one important task is to test the equality of two survival functions of treatment and control groups:

$$H_0:S_1(t)=S_2(t)\text{versus}{H}_1:S_1(t)\ne S_2(t)\text{for , some}t$$ (1)

where $S_k(t)$ is the true survival function of group $k$ ( $k=1,2$ ). In the past several decades, the log-rank test ¹ has been dominantly used and it possesses the highest statistical power under the proportional hazards (PHs) assumption, that is the hazard functions of the two groups are proportional to each other.^2,3 However, if the PH assumption is violated, which is frequently encountered in practice, the log-rank test is no longer optimal and other more powerful tests can be used to distinguish $S_1(t)$ and $S_2(t)$ , for example, a family of weighted log-rank tests. Unlike the log-rank test which weighs all events equally, the weighted log-rank tests assign different weights to different events, for example, putting more weights on early survival differences or late survival differences. With the weight function constructed on the basis of the estimated survival function of combined data ⁴ or the proportion of at-risk observations,^5,6 one can place more weights on events within a particular time period in order to detect the difference in survival functions with a higher power.

Due to censoring, the mean survival time may not be estimable and, as a result, the restricted mean survival time (RMST) has been developed as a summary statistic for the survival distribution.^7–9 The $\tau$ -RMST with respect to the survival function $S(t)$ is defined as the expected value of survival time $T$ up to a time point $\tau$ ,

$$\mu (\tau ;S)=\mathbb{E}(min\{T,\tau \})={\int }_0^{\tau }S(t)\text{dt}.$$ (2)

In contrast to the typical mean survival time which is sensitive to outliers, the RMST is robust due to the incorporation of $\tau$ in its calculation. By plugging in the Kaplan–Meier ¹⁰ (KM) estimator $\hat{S}(t)$ , a non-parametric maximum likelihood estimator (NPMLE) for the survival function, ¹¹ $\mu (\tau ;S)$ can be easily estimated as the area under $\hat{S}(t)$ from $0$ to $\tau$ , denoted as $\mu (\tau ;\hat{S})$ . To compare the survival distributions of two groups, inferences based on RMST have been extensively studied.^12–17 In recent works,^18–20 the RMST-based test has been shown to be more powerful than the log-rank test when the PH assumption is violated and the power is comparable when the PH assumption holds.

A crucial issue in the RMST-based test is the selection of the time point $\tau$ , which can be specified at the design stage or selected in a data-driven way after data collection. ²¹ Because the power of the RMST-based test highly depends on the pattern of differences between the two survival functions, an improperly chosen $\tau$ would lead to poor performance in identifying the survival differences. For example, the survival functions $S_1(t)$ and $S_2(t)$ in Figure 1 cross within the interval $[0,2]$ and the $\tau$ -RMSTs of the two groups evaluated at $\tau =2$ happens to be equal, resulting in inability of using $\mu (2;S_1)-\mu (2;S_2)$ to distinguish the survival functions. To overcome the aforementioned challenge, several non-parametric RMST-based tests have been developed to adapt to the pattern of differences between the survival functions with high robustness and flexibility. Horiguchi et al. ²² proposed the RMST-based versatile test by constructing the test statistic as the maximum of $Z$ -statistics corresponding to RMST differences $\mu ({\tau }_1;S_1)-\mu ({\tau }_1;S_2),\dots ,\mu ({\tau }_d;S_1)-\mu ({\tau }_d;S_2)$ , while Wolski et al. ²³ used the absolute difference of RMST for comparing net survival distributions. Without analytical derivation of the null distribution, all the existing methods unfortunately rely on numerical procedures, such as wild bootstrap and permutation, to make inferences and are thus computationally intensive.

Figure 1.

The true survival functions of two treatment groups whose $\tau$ -restricted mean survival time ( $\tau$ -RMST) evaluated at $\tau =2$ are equal.

Inspired by the concept of influence function ²⁴ and its relationship with the asymptotic distribution of statistical functional, ²⁵ we propose an omnibus Wald test for the hypothesis in (1) based on RMST and the KM estimator. Via a perturbation procedure, we can estimate the asymptotic covariance matrix of estimated RMSTs at multiple time points ${\tau }_1,\dots ,{\tau }_d$ for each group with higher computational efficiency than the existing method. ²⁶ We construct a Wald test statistic for a vector of RMST differences $\mu ({\tau }_1;S_1)-\mu ({\tau }_1;S_2),\dots ,\mu ({\tau }_d;S_1)-\mu ({\tau }_d;S_2)$ , which follows a ${\chi }_d^2$ distribution asymptotically. To our knowledge, our work is the first one to use an influence function for asymptotic covariance matrix estimation and omnibus hypothesis testing in survival analysis, which nicely circumvents intensive resampling-based computation. Extensive numerical studies are conducted to assess the size and power of the proposed Wald test in comparison with several existing tests. It is shown that the performance of our Wald test is stable across various cases with power comparable to the state-of-the-art tests. Particularly, when the true survival functions $S_1(t)$ and $S_2(t)$ cross, our method outperforms all the existing tests dramatically.

The remainder of this paper is organized as follows. In Section 2, we introduce the RMST as a statistical functional and derive the asymptotic distribution in terms of influence function. A perturbation procedure is further developed to compute the RMST-based Wald test statistic, along with recommendations on the selection of time points. We conduct extensive simulations in Section 3 to demonstrate the effectiveness of estimation of the asymptotic covariance matrix, validate our RMST-based Wald test and compare it with existing methods under different scenarios. In Section 4, we apply the proposed test to analyze two real datasets and illustrate its broad applicability to different cases. Section 5 concludes with a brief discussion.

2. Methodology

2.1. RMST as statistical functional

Let $T_1,\dots ,T_n$ be independent and identically distributed (i.i.d.) event times with the survival function $S(t)$ . Let $C_1,\dots ,C_n$ be i.i.d. censoring times with the survival function $G(c)$ , which are assumed to be independent of event times. Let $(X_1,{\mathrm{\Delta }}_1),\dots ,(X_n,{\mathrm{\Delta }}_n)$ denote the observed time-to-event data, where $X_i=min(T_i,C_i)$ and ${\mathrm{\Delta }}_i=I(T_i\le C_i)$ are, respectively, the observed time and the censoring indicator for subject $i$ . Denote the joint survival distribution of the event time and censoring time as $H(t,c)=S(t)G(c),$ which can be evaluated on its entire support. According to the definition in (2), the vector of RMSTs evaluated at ${\tau }_1,\dots ,{\tau }_d$ :

$$\begin{array}{rl}\mathit{\psi }(H)& ={\left(\begin{array}{ccc}\mu ({\tau }_1;S),& \dots ,& \mu ({\tau }_d;S)\end{array}\right)}^{\mathsf{T}}\\ & ={\left(\begin{array}{ccc}{\int }_{t=0}^{{\tau }_1}\{-{\int }_{c=0}^{\mathrm{\infty }}\text{d}H(t,c)\}\text{d}t,& \dots ,& {\int }_{t=0}^{{\tau }_d}\{-{\int }_{c=0}^{\mathrm{\infty }}\text{d}H(t,c)\}\text{d}t\end{array}\right)}^{\mathsf{T}}\end{array}$$

can be treated as a statistical functional of $H(t,c)$ , ²⁷ where $\text{d}H(t,c)$ is the differential of the joint survival function $H(t,c)$ . Even though RMST in (2) appears to be irrelevant to the survival function $G(c)$ of the censoring time, we still involve $G(c)$ in the statistical functional expression of RMST throughout this article for reasons detailed in Remark 1.

Let $\mathrm{\#}\{i:\text{condition}\}$ denote the number of observations that meet the specified condition after the colon and let $X_{(1)}<\dots <X_{(M)}$ be $M$ distinct observed times. Under the convention that ${\mathrm{\Delta }}_i=I(T_i\le C_i)$ , the KM estimators of $S(t)$ and $G(c)$ are, respectively, given by

$$\hat{S}(t)=\prod _{m:X_{(m)}\le t}\{1-\frac{\mathrm{\#}\{i:X_i=X_{(m)};{\mathrm{\Delta }}_i=1\}}{\mathrm{\#}\{i:X_i\ge X_{(m)}\}}\},t\le X_{(M)}$$ (3)

$$\hat{G}(c)=\prod _{m:X_{(m)}\le c}\{1-\frac{\mathrm{\#}\{i:X_i=X_{(m)};{\mathrm{\Delta }}_i=0\}}{\mathrm{\#}\{i:X_i>X_{(m)}\text{ or }(X_i,{\mathrm{\Delta }}_i)=(X_{(m)},0)\}}\},c\le X_{(M)}$$ (4)

which provide the NPMLEs of the true survival functions $S(t)$ and $G(c)$ . ²⁸ Based on the multiple time points ${\tau }_1,\dots ,{\tau }_d$ , we can obtain a vector of estimated RMSTs:

$$\mathit{\psi }(\hat{H})=(\mu ({\tau }_1;\hat{S}),\dots ,\mu ({\tau }_d;\hat{S}){)}^{\mathsf{T}}$$ (5)

where $min\{{\tau }_1,\dots ,{\tau }_d\}\ge X_{(1)}$ , $max\{{\tau }_1,\dots ,{\tau }_d\}\le X_{(M)}$ and $\hat{H}(t,c)=\hat{S}(t)\hat{G}(c)$ . ²¹ To estimate the asymptotic covariance matrix of the estimated RMSTs in (5), we utilize the influence function for the asymptotic analysis as detailed in Section 2.2.

Remark 1

Although the vector of RMSTs can also be treated as a statistical functional of the survival function $S(t)$ alone, using the influence function corresponding to $S(t)$ would underestimate the asymptotic covariance matrix of RMSTs. The reason is that the KM estimator $\hat{S}(t)$ is determined by the i.i.d. observations $(X_1,{\mathrm{\Delta }}_1),\dots ,(X_n,{\mathrm{\Delta }}_n)$ from the probabilistic measure:

$$\mathrm{P}r(X>x,\mathrm{\Delta })=\mathrm{\Delta }{\int }_{x\le t\le c}\text{d}H(t,c)+(1-\mathrm{\Delta }){\int }_{x\le c<t}\text{d}H(t,c)$$

for $x\in [0,\mathrm{\infty }),\mathrm{\Delta }\in \{0,1\}$ . This implies that the distribution of estimated RMSTs is determined by $H(t,c)$ and would vary as $G(c)$ changes even under the same $S(t)$ . This can also be seen in Example IV.3.8. of Andersen et al. ²⁹ that the asymptotic variance of the estimated RMST $\mu (\tau ;\hat{S})$ ( $\tau >0$ ) depends on the survival function of the observed time

$$Y(u)=\text{Pr}(X>u)=S(u)G(u),u>0$$

If the influence function of the statistical functional of $S(t)$ is used, we can only estimate the asymptotic covariance matrix of estimated RMSTs conditional on $\hat{G}(c)$ , which is smaller than the true asymptotic covariance matrix as shown in Section 3.1. As a result, the type I error rate of the RMST-based Wald test proposed in Section 2.3 would be inflated.

2.2. Influence function

Considering a general parameter vector $\mathit{\psi }(H)$ expressed as a statistical functional of the joint survival function $H$ , the influence function of $\mathit{\psi }$ at the point $(t,c)$ is defined as

$$\mathit{\phi }(t,c;\mathit{\psi },H)=\underset{ϵ\to 0}{lim}\frac{\mathit{\psi }((1-ϵ)H+ϵ{\delta }_{t,c})-\mathit{\psi }(H)}{ϵ}$$ (6)

where ${\delta }_{t,c}$ is the survival function of the Dirac delta distribution that places probability one at the point $(t,c)$ . As long as the statistical functional $\mathit{\psi }(H)$ is differentiable with respect to the joint survival function $H$ , the influence function $\mathit{\phi }(t,c;\mathit{\psi },H)$ measures the rate and the direction at which $\mathit{\psi }(H)$ changes when $H$ is slightly perturbed by the Dirac delta distribution ${\delta }_{t,c}$ or by a contaminated observation at $(t,c)$ . ³⁰ Thus, the NPMLE of $\mathit{\psi }(H)$ with a bounded influence function is robust because an outlier can only shift the estimator $\mathit{\psi }(\hat{H})$ in a controlled range.

Given the NPMLE $\hat{H}(t,c)$ , we can apply the Taylor expansion to $\mathit{\psi }(\hat{H})$ as follows:

$$\begin{array}{rl}\mathit{\psi }(\hat{H})-\mathit{\psi }(H)=& \int \mathit{\phi }(t,c;\mathit{\psi },H)\{\text{d}\hat{H}(t,c)-\text{d}H(t,c)\}+{\mathbf{\text{R}}}_n\\ =& \int \int \mathit{\phi }(t,c;\mathit{\psi },H)\text{d}\hat{S}(t)\text{d}\hat{G}(c)-\int \int \mathit{\phi }(t,c;\mathit{\psi },H)\text{d}S(t)\text{d}G(c)+{\mathbf{\text{R}}}_n\end{array}$$

where ${\mathbf{\text{R}}}_n$ is the remainder. Provided that

$$\mathbb{E}\{\mathit{\phi }(t,c;\mathit{\psi },H)\}=\int \int \mathit{\phi }(t,c;\mathit{\psi },H)\text{d}S(t)\text{d}G(c)=\mathbf{0}$$ (7)

and

$$\sqrt{n}{\mathbf{\text{R}}}_n\to \mathbf{0}\text{in probability}$$ (8)

it is well known that

$$\sqrt{n}\{\mathit{\psi }(\hat{H})-\mathit{\psi }(H)\}=\frac{1}{\sqrt{n}}\sum _{i=1}^n\mathit{\phi }(T_i,C_i;\mathit{\psi },H)+o_p(1)$$

If the covariance matrix

$$\begin{array}{rl}{\mathbb{V}}_{\mathit{\psi }}=& \mathbb{E}\{\mathit{\phi }(t,c;\mathit{\psi },H)\mathit{\phi }(t,c;\mathit{\psi },H{)}^{\mathsf{T}}\}\\ =& \int \int \mathit{\phi }(t,c;\mathit{\psi },H)\mathit{\phi }(t,c;\mathit{\psi },H{)}^{\mathsf{T}}\text{d}S(t)\text{d}G(c)\end{array}$$ (9)

is finite, the central limit theorem implies that

$$\frac{1}{\sqrt{n}}\sum _{i=1}^n\mathit{\phi }(T_i,C_i;\mathit{\psi },H)\stackrel{D}{⟶}\text{MVN}(\mathbf{0},{\mathbb{V}}_{\mathit{\psi }})$$

where ${\displaystyle \stackrel{D}{⟶}}$ stands for convergence in distribution, and further we have

$$\sqrt{n}\{\mathit{\psi }(\hat{H})-\mathit{\psi }(H)\}\stackrel{D}{⟶}\text{MVN}(\mathbf{0},{\mathbb{V}}_{\mathit{\psi }})$$ (10)

In practice, given the observed data $(X_1,{\mathrm{\Delta }}_1),\dots ,(X_n,{\mathrm{\Delta }}_n)$ and the KM estimators (3)–(4), the asymptotic covariance matrix ${\mathbb{V}}_{\mathit{\psi }}$ can be estimated by

$$\begin{array}{rl}{\hat{\mathbb{V}}}_{\mathit{\psi }}& =\int \int \mathit{\phi }(t,c;\mathit{\psi },\hat{H})\mathit{\phi }(t,c;\mathit{\psi },\hat{H}{)}^{\mathsf{T}}\text{d}\hat{S}(t)\text{d}\hat{G}(c)\\ & =\sum _{j=1}^{M+1}\sum _{l=1}^{M+1}[\mathit{\phi }(X_{(j)},X_{(l)};\mathit{\psi },\hat{H})\mathit{\phi }(X_{(j)},X_{(l)};\mathit{\psi },\hat{H}{)}^{\mathsf{T}}\{\hat{S}(X_{(j-1)})-\hat{S}(X_{(j)})\}\{\hat{G}(X_{(l-1)})-\hat{G}(X_{(l)})\}]\end{array}$$

where $X_{(0)}=0$ , $X_{(M+1)}=\mathrm{\infty }$ , $\hat{S}(X_{(M+1)})=\hat{G}(X_{(M+1)})=0$ and $\mathit{\phi }(t,c;\mathit{\psi },\hat{H})$ is the empirical influence function of $\mathit{\psi }$ at the point $(t,c)$ . Based on the definition in (6), the influence function $\mathit{\phi }(t,c;\mathit{\psi },H)$ equals the ratio between the change of $\mathit{\psi }$ and the probability weight $ϵ$ of distributional perturbation on $H$ at $(t,c)$ for small enough $ϵ$ . As a result, we propose a perturbation procedure to estimate the empirical influence function $\mathit{\phi }(X_{(j)},X_{(l)};\mathit{\psi },\hat{H})$ ( $j,l=1,\dots ,M+1$ ) with details given below.

• (i)Generate a pseudo observation $(X_{(m^{*})},{\mathrm{\Delta }}_{(m^{*})})$ with $X_{(m^{*})}=min(X_{(j)},X_{(l)})$ , ${\mathrm{\Delta }}_{(m^{*})}=I(X_{(j)}\le X_{(l)})$ . The pseudo observation characterizes the distributional perturbation on $\hat{H}$ at $(X_{(j)},X_{(l)})$ .
• (ii)
  Compute the KM estimators of the survival functions perturbed by the pseudo observation with weight $ϵ$ ,

  $$\begin{array}{rl}& {\hat{S}}_{ϵ}(t)=\prod _{m:X_{(m)}\le t}\{1-\frac{\mathrm{\#}\{i:X_i=X_{(m)};{\mathrm{\Delta }}_i=1\}+\frac{nϵ}{1-ϵ}I\{X_{(m^{*})}=X_{(m)};{\mathrm{\Delta }}_{(m^{*})}=1\}}{\mathrm{\#}\{i:X_i\ge X_{(m)}\}+\frac{nϵ}{1-ϵ}I\{X_{(m^{*})}\ge X_{(m)}\}}\}\\ & {\hat{G}}_{ϵ}(c)=\prod _{m:X_{(m)}\le c}\{1-\frac{\mathrm{\#}\{i:X_i=X_{(m)};{\mathrm{\Delta }}_i=0\}+\frac{nϵ}{1-ϵ}I\{X_{(m^{*})}=X_{(m)};{\mathrm{\Delta }}_{(m^{*})}=0\}}{\mathrm{\#}\{i:X_i>X_{(m)}\text{ or }(X_i,{\mathrm{\Delta }}_i)=(X_{(m)},0)\}+\frac{nϵ}{1-ϵ}I\{X_{(m^{*})}>X_{(m)}\text{ or }(X_{(m^{*})},{\mathrm{\Delta }}_i)=(X_{(m)},0)\}}\}\end{array}$$ (11)

  where $t\le X_{(M)}$ , $c\le X_{(M)}$ , and the perturbation parameter $ϵ>0$ takes a small value (e.g. $ϵ=0.001$ ).
• (iii)
  Estimate the empirical influence function at point $(X_{(j)},X_{(l)})$ by

  $$\begin{array}{rl}\hat{\mathit{\phi }}(X_{(j)},X_{(l)};\mathit{\psi },\hat{H})& =\frac{\mathit{\psi }({\hat{H}}_{ϵ})-\mathit{\psi }(\hat{H})}{ϵ}\end{array}$$ (12)

  where ${\hat{H}}_{ϵ}(t,c)={\hat{S}}_{ϵ}(t){\hat{G}}_{ϵ}(c)$ and $\mathit{\psi }({\hat{H}}_{ϵ})-\mathit{\psi }(\hat{H})$ characterizes the change of $\mathit{\psi }$ due to the distributional perturbation of weight $ϵ$ on $\hat{H}$ at $(X_{(j)},X_{(l)})$ .

By (11), the value $\hat{\mathit{\phi }}(X_{(j)},X_{(l)};\mathit{\psi },\hat{H})$ remains the same as long as the values of $X_{(m^{*})}=min(X_{(j)},X_{(l)})$ and ${\mathrm{\Delta }}_{(m^{*})}=I(X_{(j)}\le X_{(l)})$ are unchanged with respect to different pairs of $j$ and $l$ . For convenience, we define ${\hat{\mathit{\phi }}}_{X_{(m^{*})},{\mathrm{\Delta }}_{(m^{*})}}=\hat{\mathit{\phi }}(X_{(j)},X_{(l)};\mathit{\psi },\hat{H})$ for each unique pair $(X_{(m^{*})},{\mathrm{\Delta }}_{(m^{*})})$ . Given the KM estimators in (3) to (4), we have

$$\begin{array}{rl}{\hat{\mathbb{V}}}_{\mathit{\psi }}& =\sum _{m^{*}=1}^{M+1}\sum _{l=m^{*}}^{M+1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},1}^{\mathsf{T}}\{\hat{S}(X_{(m^{*}-1)})-\hat{S}(X_{(m^{*})})\}\{\hat{G}(X_{(l-1)})-\hat{G}(X_{(l)})\}\\ & +\sum _{m^{*}=1}^M\sum _{j=m^{*}+1}^{M+1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},0}{\hat{\mathit{\phi }}}_{X_{(m^{*})},0}^{\mathsf{T}}\{\hat{S}(X_{(j-1)})-\hat{S}(X_{(j)})\}\{\hat{G}(X_{(m^{*}-1)})-\hat{G}(X_{(m^{*})})\}\\ & =\sum _{m^{*}=1}^{M+1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},1}^{\mathsf{T}}\{\hat{S}(X_{(m^{*}-1)})-\hat{S}(X_{(m^{*})})\}\hat{G}(X_{(m^{*}-1)})\\ & +\sum _{m^{*}=1}^{M+1}{\hat{\mathit{\phi }}}_{X_{(m^{*})},0}{\hat{\mathit{\phi }}}_{X_{(m^{*})},0}^{\mathsf{T}}\hat{S}(X_{(m^{*})})\{\hat{G}(X_{(m^{*}-1)})-\hat{G}(X_{(m^{*})})\}\end{array}$$ (13)

Compared with the existing formula, ²⁶ our perturbation procedure can estimate the asymptotic covariance matrix at a much lower cost. In the work of Murray and Tsiatis, ²⁶ the asymptotic covariance of $\mu ({\tau }_a;\hat{S})$ and $\mu ({\tau }_b;\hat{S})$ ( $1\le a\le b\le d$ ) is estimated as

$$\begin{array}{r}\sum _{m:X_{(m)}<{\tau }_a}\frac{\{\mu ({\tau }_a;\hat{S})-\mu (X_{(m)};\hat{S})\}\{\mu ({\tau }_b;\hat{S})-\mu (X_{(m)};\hat{S})\}}{\hat{G}(X_{(m-1)})\hat{S}(X_{(m-1)})}\frac{\mathrm{\#}\{i:X_i=X_{(m)};{\mathrm{\Delta }}_i=1\}}{\mathrm{\#}\{i:X_i\ge X_{(m)}\}}\end{array}$$

with computational complexity $\mathcal{O}(n^3)$ and as a result, at least $\mathcal{O}(n^3{d}^2)$ operations are needed to obtain ${\hat{\mathbb{V}}}_{\mathit{\psi }}$ . If a (wild) bootstrap procedure with $B$ bootstraps is adopted, the computation cost can be reduced to $O(Bnd)$ . On the contrary, the total computational complexity of the perturbation procedure and equation (13) is $\mathcal{O}(n^{2}d)$ . If the number of bootstraps $B$ is set at a value larger than the sample size $n$ , our method would be more computationally efficient.

Remark 2

If all the observed times $X_1,\dots ,X_n$ are different (i.e. $M=n$ ), we have

$$\begin{array}{rl}\{\hat{S}(X_{(m^{*}-1)})-\hat{S}(X_{(m^{*})})\}\hat{G}(X_{(m^{*}-1)})& =\frac{{\mathrm{\Delta }}_{(m^{*})}}{n}\\ \hat{S}(X_{(m^{*})})\{\hat{G}(X_{(m^{*}-1)})-\hat{G}(X_{(m^{*})})\}& =\frac{1-{\mathrm{\Delta }}_{(m^{*})}}{n}\end{array}$$

for $m^{*}=1,\dots ,n$ . As a result, the estimated asymptotic covariance matrix reduces to

$$\begin{array}{rl}{\hat{\mathbb{V}}}_{\mathit{\psi }}=& \frac{1}{n}\sum _{m^{*}=1}^n{\hat{\mathit{\phi }}}_{X_{(m^{*})},{\mathrm{\Delta }}_{(m^{*})}}{\hat{\mathit{\phi }}}_{X_{(m^{*})},{\mathrm{\Delta }}_{(m^{*})}}^{\mathsf{T}}\end{array}$$

which equals the Jackknife estimator of the covariance matrix of $\hat{\mathit{\psi }}$ ³¹ multiplied by the sample size $n$ if $ϵ=-1/(n-1)$ is used to compute the empirical influence function.

2.3. RMST-Based Wald test

Now we consider a vector of RMSTs at the pre-specified time points ${\tau }_1,\dots ,{\tau }_d$ , $\mathit{\psi }(H)=(\mu ({\tau }_1,S),\dots ,\mu ({\tau }_d,S){)}^{\mathsf{T}}$ . By the linearity of RMSTs with respect to the joint survival function, we have

$$\begin{array}{rl}\mathit{\phi }(t,c;\mathit{\psi },H)=& \underset{ϵ\to 0}{lim}\frac{\mathit{\psi }((1-ϵ)H+ϵ{\delta }_{t,c})-\mathit{\psi }(H)}{ϵ}\\ =& \underset{ϵ\to 0}{lim}\frac{ϵ\mathit{\psi }({\delta }_{t,c})-ϵ\mathit{\psi }(H)}{ϵ}\\ =& \mathit{\psi }({\delta }_{t,c})-\mathit{\psi }(H)\\ =& (min\{t,{\tau }_1\},\dots ,min\{t,{\tau }_d\}{)}^{\mathsf{T}}-\mathit{\psi }(H)\end{array}$$

which falls into a compact subset of ${\mathcal{R}}^d$ and thus equations (7) and (9) hold. The boundedness of $\mathit{\phi }(t,c;\mathit{\psi },H)$ suggests the robustness of the estimated RMST to outliers, in contrast to the usual belief that the mean is sensitive to outliers. In addition, we have that

$$\begin{array}{rl}{\mathbf{\text{R}}}_n& =\mathit{\psi }(\hat{H})-\mathit{\psi }(H)-\int \int \mathit{\phi }(t,c;\mathit{\psi },H)\text{d}\hat{H}(t,c)\\ & =\mathit{\psi }(\hat{H})-\mathit{\psi }(H)-\int \int (min\{t,{\tau }_1\},\dots ,min\{t,{\tau }_S\}{)}^{\mathsf{T}}\text{d}\hat{S}(t)\text{d}\hat{G}(c)+\mathit{\psi }(H)\\ & =\mathbf{0}\end{array}$$

implying that equation (8) holds. As a result, the asymptotic distribution in (10) holds for the vector of estimated RMSTs in (5).

Consider the i.i.d. observations $(X_{k1},{\mathrm{\Delta }}_{k1}),\dots ,(X_{k{n}_k},{\mathrm{\Delta }}_{k{n}_k})$ for group $k$ ( $k=1,2$ ) with sample size $n_k$ . Let $S_k(t)$ and $G_k(c)$ denote the true survival functions for the event time and censoring time, respectively. If $min\{S_1({\tau }_a),S_2({\tau }_a)\}<1$ and $max\{H_1({\tau }_a,{\tau }_a),H_2({\tau }_a,{\tau }_a)\}>0$ for all $p=1,\dots ,d$ , it is straightforward to show from (10) that as $min\{n_1,n_2\}\to \mathrm{\infty }$ with $n_1/(n_1+n_2)\to \kappa \in (0,1)$ :

$$\{\mathit{\psi }({\hat{H}}_1)-\mathit{\psi }({\hat{H}}_2)\}-\{\mathit{\psi }(H_1)-\mathit{\psi }(H_2)\}\stackrel{D}{⟶}\text{MVN}(\mathbf{0},\frac{{\mathbb{V}}_{1,\mathit{\psi }}}{n_1}+\frac{{\mathbb{V}}_{2,\mathit{\psi }}}{n_2})$$ (14)

where ${\hat{H}}_k(t,c)={\hat{S}}_k(t){\hat{G}}_k(c)$ for $k=1,2$ . As a result, we can construct an RMST-based Wald test statistic for the hypothesis in (1),

$$T_{\mathit{\psi }}=(\mathit{\psi }({\hat{H}}_1)-\mathit{\psi }({\hat{H}}_2){)}^{\mathsf{T}}(\frac{{\hat{\mathbb{V}}}_{1,\mathit{\psi }}}{n_1}+\frac{{\hat{\mathbb{V}}}_{2,\mathit{\psi }}}{n_2}{)}^{-1}(\mathit{\psi }({\hat{H}}_1)-\mathit{\psi }({\hat{H}}_2))$$ (15)

where ${\hat{\mathbb{V}}}_{k,\mathit{\psi }}$ is obtained by implementing the perturbation procedure and (13) in Section 2.2 on observations of group $k$ ( $k=1,2$ ). Based on (14), the Wald test statistic $T_{\mathit{\psi }}$ follows the ${\chi }_d^2$ distribution asymptotically when the null hypothesis in (1) is true.

Remark 3

Although the RMST-based Wald test is developed under the simple random censorship model $H(t,s)=S(t)G(c)$ , it is also valid for the event-driven censoring model with staggered entry, ³² where censoring times $C_1,\dots ,C_n$ are shown to imply no information on i.i.d. event times $T_1,\dots ,T_n$ and thus RMSTs ${\left(\begin{array}{ccc}\mu ({\tau }_1;S),& \dots ,& \mu ({\tau }_d;S)\end{array}\right)}^{\mathsf{T}}$ . As a result, using KM estimators (3) and (4) for inference on the study time scale can be regarded as the composite marginal likelihood method ³³ under the event-driven censoring model with staggered entry, where the perturbation procedure and (13) in Section 2.2 can accurately estimate the asymptotic covariance matrix of RMSTs and our RMST-based Wald test is thus valid.

2.4. Selection of time points

When the null hypothesis in (1) is not true, the RMST-based Wald test with a larger number of time points ( $d$ ) is asymptotically more powerful than that with a smaller $d$ . However, in practice where the number of observations is finite, a suitable $d$ should be chosen because the power of the RMST-based Wald test would increase but eventually decrease as $d$ continues to increase. More severely, the probability of the type I error would be inflated if $d$ is so large that the asymptotic null distribution $T_{\mathit{\psi }}\sim {\chi }_d^2$ does not hold. This can be seen from Figure 2, where the size and power of the RMST-based Wald test are exhibited with different values of $d$ . Here, we fix the sample size $n=200$ for both groups, the perturbation parameter $ϵ=0.001$ , and the significance level $\alpha =0.05$ . We use the rejection proportions in $1000$ simulated datasets under the null case where the survival functions of two groups are the same and the non-null case in Figure 1 to empirically evaluate the size and power of the RMST-based Wald test, respectively. Generally speaking, the number of time points ( $d$ ) should be chosen based on clinical knowledge, the total sample size, censoring rate, available computational resources, etc. We recommend $d$ to be between $4$ and $6$ as a tradeoff between statistical power and computational cost.

Figure 2.

The rejection proportions of the restricted mean survival time (RMST)-based Wald test under different numbers of time points ( $d=2$ to 32) with sample size $n=200$ for both groups and perturbation parameter $ϵ=0.001$ . We consider the null case $H_0$ where the survival functions of two groups are the same and the non-null case $H_1$ as given in Figure 1.

In comparative studies, it is suggested that the time points ( ${\tau }_1,\dots ,{\tau }_d$ ) should be pre-specified at the design stage based on prior clinical information.^19,21 If it is needed to select the time points ${\tau }_1,\dots ,{\tau }_d$ in a data-driven way, which is usually the case in observational studies, we suggest using the equal-distance $\{1/d,2/d,\dots ,1\}$ quantiles of all the observed event times $\{X_{ki}:{\mathrm{\Delta }}_{ki}=1,k=1,2\}$ as the time points ${\tau }_1,\dots ,{\tau }_d$ . The reason is that the estimated correlation between $\mu ({\tau }_a,S)$ and $\mu ({\tau }_b,S)$ ( $1\le a<b\le d$ ) would be large if there are too few events observed in the time interval $[{\tau }_a,{\tau }_b]$ . Such high correlation would lead to strong collinearity in the estimated asymptotic covariance matrices ${\hat{\mathbb{V}}}_{k,\mathit{\psi }}$ ( $k=1,2$ ) and thus incur power loss for the RMST-based Wald test.

3. Simulations

3.1. Estimation accuracy of ${\hat{\mathbb{V}}}_{\mathit{\psi }}$

As the estimated asymptotic covariance matrix ${\hat{\mathbb{V}}}_{\mathit{\psi }}$ computed by the perturbation procedure and (13) plays a crucial role in the Wald test statistic $T_{\mathit{\psi }}$ , we check whether it is a valid estimator of the true asymptotic covariance matrix ${\mathbb{V}}_{\mathit{\psi }}$ on $1000$ simulated datasets under different sample sizes $n=100,200,500$ and $1000$ . In each dataset, the i.i.d. observations $(X_1,{\mathrm{\Delta }}_1),\dots ,(X_n,{\mathrm{\Delta }}_n)$ are generated, where $X_i=min(T_i,C_i,C_0)$ and ${\mathrm{\Delta }}_i=I(T_i\le min\{C_i,C_0\})$ with the event time $T_i\sim S(t)=\exp (-t/2)$ , the individual censoring time $C_i\sim G(c)=\exp (-c/4)$ and the administrative censoring time $C_0=2$ ( $i=1,\dots ,n$ ). Let $\mathit{\psi }({\hat{H}}^{(b)})$ and ${\hat{\mathbb{V}}}_{\mathit{\psi }}^{(b)}$ denote the vector of estimated RMSTs and the estimated asymptotic covariance matrix in the $b$ th dataset ( $b=1,\dots ,1000$ ). We measure the estimation accuracy of ${\hat{\mathbb{V}}}_{\mathit{\psi }}$ by the average relative Frobenius norm ( $\text{ARFN}$ ) of ${\hat{\mathbb{V}}}_{\mathit{\psi }}^{(1)},\dots ,{\hat{\mathbb{V}}}_{\mathit{\psi }}^{(1000)}$ as follows:

$$\text{ARFN}=\frac{1}{1000}\sum _{b=1}^{1000}\frac{\Vert {\hat{\mathbb{V}}}_{\mathit{\psi }}^{(b)}-{\overline{\mathbb{V}}}_{\mathit{\psi }}{\Vert }_{\text{F}}}{\Vert {\overline{\mathbb{V}}}_{\mathit{\psi }}{\Vert }_{\text{F}}}$$

where $\Vert \cdot {\Vert }_{\text{F}}$ is the Frobenius norm of matrices, and

$$\begin{array}{rl}{\overline{\mathbb{V}}}_{\mathit{\psi }}& =\frac{1}{1000}\sum _{b=1}^{1000}n\{\mathit{\psi }({\hat{H}}^{(b)})-\overline{\mathit{\psi }}(\hat{H})\}\{\mathit{\psi }({\hat{H}}^{(b)})-\overline{\mathit{\psi }}(\hat{H}){\}}^{\mathsf{T}}\\ \overline{\mathit{\psi }}(\hat{H})& =\frac{1}{1000}\sum _{b=1}^{1000}\mathit{\psi }({\hat{H}}^{(b)})\end{array}$$ (16)

Table 1 presents the $\text{ARFN}$ of ${\hat{\mathbb{V}}}_{\mathit{\psi }}^{(1)},\dots ,{\hat{\mathbb{V}}}_{\mathit{\psi }}^{(1000)}$ under different sample sizes ( $n=100,200,500$ , and $1000$ ), different perturbation parameters ( $ϵ=0.0001,0.001$ , and $0.01$ ), three sets of time points ( $\{2\}$ , $\{1,2\}$ and $\{0.5,1,1.5,2\}$ ). It is clear that the choice of $ϵ$ does not have a substantial impact on the difference between ${\hat{\mathbb{V}}}_{\mathit{\psi }}$ and the true covariance matrix. The estimation accuracy of ${\hat{\mathbb{V}}}_{\mathit{\psi }}$ improves as the sample size increases, implying the effectiveness of the proposed procedure in estimating the asymptotic covariance matrix.

Table 1.

Estimation accuracy (measured in ARFN for which the smaller the better) of the estimated asymptotic covariance matrix under different sample sizes ( $n$ ), different perturbation parameters ( $ϵ$ ) and different sets of time points.

	Time points	$n=100$	$n=200$	$n=500$	$n=1000$
Treating RMST as statistical functional of $H(t,c)$
$ϵ=0.0001$	$\{2\}$	0.0623	0.0488	0.0331	0.0206
	$\{1,2\}$	0.0740	0.0521	0.0380	0.0246
	$\{0.5,1,1.5,2\}$	0.0781	0.0566	0.0393	0.0256
$ϵ=0.001$	$\{2\}$	0.0621	0.0509	0.0341	0.0201
	$\{1,2\}$	0.0761	0.0546	0.0385	0.0258
	$\{0.5,1,1.5,2\}$	0.0808	0.0574	0.0394	0.0276
$ϵ=0.01$	$\{2\}$	0.0632	0.0515	0.0300	0.0199
	$\{1,2\}$	0.0767	0.0588	0.0354	0.0263
	$\{0.5,1,1.5,2\}$	0.0811	0.0597	0.0367	0.0278
Treating RMST as statistical functional of $S(t)$
$ϵ=0.001$	$\{2\}$	0.4750	0.4547	0.4618	0.4558
	$\{1,2\}$	0.4466	0.4260	0.4349	0.4268
	$\{0.5,1,1.5,2\}$	0.4159	0.3928	0.4056	0.3953

ARFN: average relative Frobenius norm; RMST: restricted mean survival time.

For comparison, we also evaluate the accuracy of the estimated asymptotic covariance matrix when the vector of RMSTs is treated as a statistical functional of $S(t)$ alone. From Tables 1 and 2, it is clear that treating the vector of RMSTs as a statistical functional of the survival function $S(t)$ would underestimate the asymptotic covariance matrix of the estimated RMSTs. We also examine whether the same pattern exists on $1000$ simulated datasets under the event-driven censorship with staggered entry and sample size $n=200$ . For each dataset, the i.i.d. event times $T_1,\dots ,T_{200}\sim S(t)=\exp (-t/2)$ and entry times $R_1,\dots ,R_{200}\sim \text{Unif}(0,2)$ are generated. With a fixed censoring rate $50\mathrm{\%}$ , we let the event-driven censoring time $C^{*}$ (on the calendar scale) be the $100$ th smallest value in $\{T_i+R_i;i=1,\dots ,200\}$ and then observations $(X_1,{\mathrm{\Delta }}_1),\dots ,(X_{200},{\mathrm{\Delta }}_{200})$ are generated in the way that $X_i=min(T_i,C^{*}-R_i)$ and ${\mathrm{\Delta }}_i=min(T_i\le C^{*}-R_i)$ ( $i=1,\dots ,200$ ). From Table 2, it is clear that under the event-driven censorship with staggered entry, our perturbation procedure can still accurately estimate the asymptotic covariance matrix of the estimated RMSTs as stated in Remark 3 while treating the vector of RMSTs as a statistical functional of the survival function $S(t)$ alone would underestimate the asymptotic covariance matrix.

Table 2.

Average value of the estimated asymptotic covariance matrix under the random censorship or event-driven censorship with staggered entry, sample size $n=200$ , the perturbation parameter $ϵ=0.001$ and time points $\{0.5,1,1.5,2\}$ when the vector of RMSTs is treated as a statistical functional of $H(t,c)$ or $S(t)$ .

		Event-driven censorship
	Random censorship	with staggered entry
${\overline{\mathbb{V}}}_{\mathit{\psi }}$ in (16)	$\left(\begin{array}{cccc}0.018& 0.039& 0.055& 0.067\\ 0.039& 0.112& 0.179& 0.230\\ 0.055& 0.179& 0.327& 0.448\\ 0.067& 0.230& 0.448& 0.665\end{array}\right)$	$\left(\begin{array}{cccc}0.017& 0.038& 0.055& 0.067\\ 0.038& 0.110& 0.171& 0.219\\ 0.055& 0.171& 0.300& 0.406\\ 0.067& 0.219& 0.406& 0.598\end{array}\right)$
$\frac{1}{1000}\sum _{b=1}^{1000}{\hat{\mathbb{V}}}_{\mathit{\psi }}^{(b)}$ when RMST treated as statistical functional of $H(t,c)$	$\left(\begin{array}{cccc}0.017& 0.039& 0.056& 0.069\\ 0.039& 0.116& 0.184& 0.236\\ 0.056& 0.184& 0.330& 0.452\\ 0.069& 0.236& 0.452& 0.666\end{array}\right)$	$\left(\begin{array}{cccc}0.016& 0.036& 0.052& 0.063\\ 0.036& 0.104& 0.163& 0.209\\ 0.052& 0.163& 0.288& 0.392\\ 0.063& 0.209& 0.392& 0.581\end{array}\right)$
$\frac{1}{1000}\sum _{b=1}^{1000}{\hat{\mathbb{V}}}_{\mathit{\psi }}^{(b)}$ when RMST treated as statistical functional of $S(t)$	$\left(\begin{array}{cccc}0.015& 0.031& 0.042& 0.048\\ 0.031& 0.085& 0.124& 0.146\\ 0.042& 0.124& 0.200& 0.245\\ 0.048& 0.146& 0.245& 0.315\end{array}\right)$	$\left(\begin{array}{cccc}0.015& 0.032& 0.044& 0.051\\ 0.032& 0.089& 0.132& 0.158\\ 0.044& 0.132& 0.215& 0.271\\ 0.051& 0.158& 0.271& 0.358\end{array}\right)$

3.2. Power Comparisons

We conduct simulations to evaluate the performance of our RMST-based Wald test in various cases and compare it with the log-rank test and the RMST-based versatile test, ²² which is more similar to the confidence band test. Although the versatile test also tests the null hypothesis in (1) based on RMSTs, Horiguchi et al. ²² used a wild bootstrap procedure to approximate the null distribution of the test statistic

$$\underset{k\in \{1,\dots ,d\}}{max}|\frac{\sqrt{n_1+n_0}\{\mu ({\tau }_k;{\hat{S}}_1)-\mu ({\tau }_k;{\hat{S}}_0)\}}{{\hat{\sigma }}_k}|$$

where ${\hat{\sigma }}_k$ is the estimated standard error of the numerator, and compute the $p$ -value.

We first validate the size of our RMST-based Wald test in three null cases where the survival functions for two groups are the same. We then consider six different types of non-null cases to compare our method with the existing ones in distinguishing various types of differences between survival functions. The true survival functions $S_1(t)$ and $S_2(t)$ of nine cases are listed as follows.

• -
  Null cases:

  • (i)
    $S_1(t)=S_2(t)=\exp (-t/2)$ .
  • (ii)
    $S_1(t)=S_2(t)$ both from $\text{Weibull}(2,0.5)$ .
  • (iii)
    $S_1(t)=S_2(t)$ both from $\text{Weibull}(2,2)$ .
• -
  Non-null cases:

  1. Proportional hazards: $S_1(t)=\exp (-t/2)$ and

     $$S_2(t)=\exp \{-(1+\lambda )t/2\}$$

     where $\lambda =0.1,\dots ,1$ .
  2. Crossing hazards: $S_1(t)=\exp (-t/2)$ and

     $$S_2(t)=\exp \{-(1+\lambda )t/2+\lambda max(t-1,0)\}$$

     where $\lambda =0.1,\dots ,1$ .
  3. Crossing survival functions: $S_1(t)=\exp (-t/2)$ and

     $$S_2(t)=\lambda (1-t/e)+(1-\lambda )\exp (-t/2)$$

     where $\lambda =0.1,\dots ,1$ .
  4. Early difference: $S_1(t)=(1+t{)}^{-1}$ and

     $$S_2(t)=2(1+t{)}^{-1}(2+min\{t,t_0\}{)}^{-1}$$

     where $t_0=0.1,\dots ,1$ .
  5. Middle difference: $S_1(t)=(1+t{)}^{-1}$ and

     $$S_2(t)=2(1+t{)}^{-1}\{2+min(t,1+t_0/2)-min(t,1-t_0/2){\}}^{-1}$$

     where $t_0=0.1,\dots ,1$ .
  6. Late difference: $S_1(t)=(1+t{)}^{-1}$ and

     $$S_2(t)=2(1+t{)}^{-1}(2+max\{t+t_0-2,0\}{)}^{-1}$$

     where $t_0=0.1,\dots ,1$ .

The survival functions $S_1(t)$ and $S_2(t)$ with $\lambda =0.5,1$ under non-null cases (a) to (c) are shown in the top panel of Figure 3 and those with $t_0=0.5,1$ under non-null cases (d) to (f) are shown in the bottom panel. In non-null cases (a) to (c), the parameter $\lambda$ controls the size of the gap between $S_1(t)$ and $S_2(t)$ , where a larger value of $\lambda$ leads to a larger gap, and $S_1(t)$ equals $S_2(t)$ when $\lambda =0$ . In non-null cases (d) to (f), the parameter $t_0$ is the length of the time period in $[0,2]$ where the hazard functions of two groups are different, and $S_1(t)$ equals $S_2(t)$ when $t_0=0$ .

Figure 3.

The true survival functions $S_1(t)$ and $S_2(t)$ under six different non-null cases and different values of $\lambda =0.5$ , $1$ or $t_0=0.5$ , $1$ . (a) Proportional hazards; (b) crossing hazards; (c) crossing survival functions; (d) early difference; (e) middle difference; and (f) late difference.

For each null case, we simulate 1000 datasets to validate the size of hypothesis testing methods under different sample sizes $n=100$ , $200$ , $500$ , and $1000$ for both groups. For each dataset, we simulate the i.i.d. observations $(X_{k1},{\mathrm{\Delta }}_{k1}),\dots ,(X_{kn},{\mathrm{\Delta }}_{kn})$ for group $k$ ( $k=1,2$ ), where $X_{ki}=min(T_{ki},C_{ki},C_0)$ and ${\mathrm{\Delta }}_{ki}=I(T_{ki}\le min\{C_{ki},C_0\})$ with the event time $T_{ki}\sim S_k(t)$ , the individual censoring time $C_{ki}\sim G(c)=\exp \{-c/4\}$ and the administrative censoring time $C_0=2$ for $i=1,\dots ,n$ . We apply the RMST-based Wald test (15) with three sets of time points $\{2\}$ , $\{1,2\}$ and $\{0.5,1,1.5,2\}$ , the perturbation parameter $ϵ=0.001$ and the significance level $\alpha =0.05$ . We also conduct the log-rank test and the RMST-based versatile test ²² with the same sets of time points for comparison. The rejection proportions of our test and the existing approaches under different null cases and sample sizes are presented in Table 3. The rejection proportions of our RMST-based Wald test (15) are all around the nominal level $5\mathrm{\%}$ , implying that the size of our RMST-based Wald test can be maintained at the significance level when $H_0$ is true, regardless of how many and which time points are chosen.

Table 3.

The rejection proportions ( $\mathrm{\%}$ ) of the log-rank test, the RMST-based versatile test, and our RMST-based Wald test with different sets of time points, different null cases, and sample sizes ( $n$ ) over 1000 simulated datasets.

	Null case (i)				Null case (ii)				Null case (iii)
$n$	$100$	$200$	$500$	$1000$	$100$	$200$	$500$	$1000$	$100$	$200$	$500$	$1000$
Log-rank	5.1	4.4	4.5	5.5	5.2	4.2	2.9	5.2	4.9	3.4	3.7	5.1
${\text{Versatile}}_{\{2\}}$	5.7	5.1	4.3	5.0	5.5	5.0	3.1	5.7	4.7	3.9	3.9	5.5
${\text{Versatile}}_{\{1,2\}}$	5.2	5.2	4.0	5.5	4.8	4.8	3.9	5.5	5.7	4.2	4.4	5.1
${\text{Versatile}}_{\{0.5,1,1.5,2\}}$	5.5	5.0	4.4	5.8	4.7	4.2	3.9	5.5	5.3	4.8	5.0	4.6
${\text{Wald}}_{\{2\}}$	5.6	5.4	4.6	5.3	5.5	4.8	3.4	5.7	4.3	3.9	4.1	5.3
${\text{Wald}}_{\{1,2\}}$	4.9	5.1	4.1	5.6	4.6	5.0	5.2	6.0	6.2	5.0	4.7	5.2
${\text{Wald}}_{\{0.5,1,1.5,2\}}$	4.3	4.1	4.8	5.4	6.4	6.3	5.6	4.4	7.1	5.1	5.7	5.1

We perform the proposed RMST-based Wald test (15) on 1000 simulated datasets with a fixed value of $\lambda$ for non-null cases (a) to (c) and a fixed value of $t_0$ for non-null cases (d) to (f) under different sample sizes $n$ for both groups to examine how the power of our test varies as the sample size increases. The parameter $\lambda$ is set as $0.5$ under non-null cases (a) and (b) and $1$ under non-null case (c), while $t_0$ is fixed as $0.5$ under non-null cases (d) to (f). The rejection proportions of our omnibus Wald test, the log-rank test, and the versatile test under different non-null cases and $n$ are exhibited in Figure 4. The true RMSTs of both groups at time point 2 are the same for all values of $\lambda$ in the case (c) of crossing survival functions, making the rejection proportions (or the type I error rates) of our RMST-based Wald test and the RMST-based versatile test with a single time point $2$ equal to the significance level $0.05$ . Except for this scenario, the power of all the tests increases toward $1$ as sample size $n$ increases.

Figure 4.

The rejection proportions of the log-rank test, the restricted mean survival time (RMST)-based versatile test, and our RMST-based Wald test versus sample size $n$ under different non-null cases. (a) Proportional hazards $\lambda =0.5$ ; (b) crossing hazards $\lambda =0.5$ ; (c) crossing survival $\lambda =1$ ; (d) early difference $t_0=0.5$ ; (e) middle difference $t_0=0.5$ ; and (f) late difference $t_0=0.5$ .

To evaluate how the power of the RMST-based Wald test (15) varies with respect to different sizes of gaps between survival functions $S_1(t)$ and $S_2(t)$ , we test $H_0:S_1(t)=S_2(t)$ on 1000 simulated datasets with a fixed sample size $n$ for both groups under different values of $\lambda$ for non-null cases (a) to (c). The sample size $n$ is $200$ under non-null cases (a) and (b) and $1000$ under the non-null case (c). The rejection proportions of our test and the existing approaches under different non-null cases and $\lambda$ are presented in Figure 5(a) to (c). Under the case of PHs, it is clear that the log-rank test possesses the highest power, which is consistent with the fact that the log-rank test is the most powerful test under the PH assumption. However, both the power values of our RMST-based Wald test (15) and the RMST-based versatile test are close to that of the log-rank test, especially when we use a single time point $2$ . On the contrary, in the cases of crossing hazards and crossing survival functions where the PH assumption does not hold, our test dominates the log-rank test and the RMST-based versatile test significantly, especially when we use multiple time points. In addition, the performance of our RMST-based Wald test with multiple time points is stable across various cases, suggesting its robustness when multiple time points are included in the omnibus test.

Figure 5.

The rejection proportions of the log-rank test, the restricted mean survival time (RMST)-based versatile test, and our RMST-based Wald test versus $\lambda$ or $t_0$ under different non-null cases. (a) Proportional hazards $n=200$ ; (b) crossing hazards $n=200$ ; (c) crossing survival $n=1000$ ; (d) early difference $n=200$ ; (e) middle difference $n=200$ ; and (f) late difference $n=200$ .

We also study the power of the RMST-based Wald test in detecting early, middle, or late difference by testing $H_0:S_1(t)=S_2(t)$ on 1000 simulated datasets with a fixed sample size of $n=200$ for both groups under different values of $t_0$ for non-null cases (d) to (f). The rejection proportions of our test and the existing methods under different cases and different values of $t_0$ are presented in Figure 5(d) to (f). Under the case of early difference, it is clear that the RMST-based versatile test possesses the highest power while the log-rank test yields the worst performance. Although our RMST-based Wald test is not the most powerful one, its power is comparable to the RMST-based versatile test. Under the cases of middle and late differences, our RMST-based Wald test with multiple time points is the most powerful one, suggesting its sensitivity to the difference between $S_1(t)$ and $S_2(t)$ regardless of the period where the difference exists. In contrast, the power of the RMST-based versatile test is even lower than that of the log-rank test under the cases of middle and late differences, suggesting that the RMST-based versatile test weighs more on early events. Therefore, our method is not inferior to the existing ones for detecting early survival differences and outperforms them in detecting middle and late survival differences. In summary, our RMST-based Wald test is much more stable and powerful across all cases at a minor cost of efficiency under some particular scenarios.

3.3. Robustness to censoring patterns

We also change the censoring distribution $G(c)$ to investigate how the RMST-based Wald test performs under different censoring patterns. We consider the null case (i) and non-null cases (a) and (c) with a fixed sample size $n=200$ for both groups. The value of $\lambda$ is set to be $0.5$ and $1$ in non-null cases (a) and (c), respectively. We use three censoring distributions $G(c)$ to generate individual censoring times to obtain different censoring rates as follows:

• - $G(c)=1-c/3,$ with $c<3$ , to yield a censoring rate of $55\mathrm{\%}$ ;
• - $G(c)=\exp (-c/4)$ to yield a censoring rate of $45\mathrm{\%}$ ;
• - $G(c)$ from $\text{Weibull}(2,4)$ to yield a censoring rate of $35\mathrm{\%}$ .

Table 4 summarizes the rejection proportions of our test at the significance level $\alpha =0.05$ with different sets of time points under different (null and non-null) cases and censoring rates. On the one hand, the size of the RMST-based Wald test is stable and centered around the significance level, implying that our test remains effective as the censoring rate increases. On the other hand, when $H_0$ is not true, the power of the RMST-based Wald test increases as the censoring rate decreases. In the PH situation, the power of the RMST-based Wald test with a single time point is the highest. When the PH assumption does not hold, it is better to use multiple time points in our RMST-based Wald test to capture the complex patterns of survival differences.

Table 4.

The rejection proportions ( $\mathrm{\%}$ ) of the restricted mean survival time (RMST)-based Wald test with different sets of time points in 1000 simulated datasets under different cases and censoring rates.

		Censoring rate
	Time points	$55\mathrm{\%}$	$45\mathrm{\%}$	$35\mathrm{\%}$
Null case (i)	$\{2\}$	5.4	5.4	5.3
	$\{1,2\}$	5.3	5.1	4.6
	$\{0.5,1,1.5,2\}$	5.2	4.1	3.5
Non-null case (a) (Proportional hazards)	$\{2\}$	82.8	84.7	87.9
	$\{1,2\}$	74.1	78.1	81.7
	$\{0.5,1,1.5,2\}$	65.7	69.3	75.9
Non-null case (c) (Crossing survival functions)	$\{2{\}}^{†}$	5.2	5.1	4.9
	$\{1,2\}$	29.7	34.8	34.1
	$\{0.5,1,1.5,2\}$	32.0	41.6	47.1

$†$ : In a non-null case (c), the true RMSTs of both groups at time point 2 are the same, making the power (or the type I error rate) of the RMST-based Wald test with a single time point $2$ equal to the significance level $0.05$ .

4. Real data analysis

To illustrate the empirical performance of our RMST-based Wald test under real situations, we consider two data examples described as follows:

• -Gastric cancer dataset^34,35: The gastric cancer dataset contains the records of $4,069$ patients with advanced/recurrent gastric cancer from $20$ randomized trials, which studied whether chemotherapy could improve patients’ overall survival and progression-free survival. In our analysis, we used the data of the $12$ th trial with 279 observations (174 in the treatment group and 105 in the control group) to test whether the progression-free survival functions of the treatment group and the control group differed or not.
• -Bone marrow transplant dataset ³⁶ : The bone marrow transplant dataset includes the time-to-event data of 101 patients with acute myelogenous leukemia in a study of bone marrow transplantation. In this dataset, $50$ patients received allogeneic bone marrow transplantation while the other $51$ patients received autologous transplantation. The event of interest in this study was death or relapse of leukemia, and the target of the study was to investigate the impact of transplant types on patients’ leukemia-free survival.

Figure 6 displays the KM estimators of the survival functions of two groups in two real data examples. In the gastric cancer dataset, it is clear that the estimated survival function of the treatment group stays higher than that of the control group for a long period of time till about 500 days, suggesting that chemotherapy is possibly effective in improving patients’ progression-free survival. However, due to the existence of several patients in the control group with extremely long progression-free survival, the estimated survival functions of the two groups cross at around $500$ days and the estimated survival rate of the control group dominates the treatment group afterward. The same phenomenon also appears in the bone marrow transplant dataset, where compared with the allogeneic group, the estimated survival rate of the autologous group is higher in the early period of follow-up but drops lower after around $15$ months.

Figure 6.

Kaplan–Meier estimators of the survival functions of two groups in (a) the gastric cancer dataset and (b) the bone marrow transplant dataset.

In the two real data examples, we apply both our RMST-based Wald test (with the perturbation parameter $ϵ=0.001$ ) and the existing RMST-based versatile test with the same set of $d=6$ time points to compare their performances in detecting the differences between two survival functions. Here, the time points ${\tau }_1,\dots ,{\tau }_6$ are selected as $1/6,2/6,\dots ,1$ quantiles of all the observed event times $\{X_{ki}:{\mathrm{\Delta }}_{ki}=1,k=1,2\}$ in each real dataset. We also apply the log-rank test as a baseline for comparison and the significance level is set at $0.05$ . For the gastric cancer dataset, both the Wald test ( $P$ -value =  $0.045$ ) and the versatile test ( $P$ -value =  $0.003$ ) results are significant while the log-rank test does not show statistical significance ( $P$ -value =  $0.055$ ). The reason may be that the PH assumption is violated in the gastric cancer dataset with a crossing in the estimated survival functions. However, in the bone marrow transplant dataset where the estimated survival functions of the two groups also cross, our RMST-based Wald test reports significant differences in the survival functions ( $P$ -value =  $0.042$ ), while both existing tests fail to show statistical significance ( $P$ -values: log-rank $0.537$ ; versatile test $0.211$ ). One possible reason for the significant result of our RMST-based Wald test is that our method is more sensitive to the late difference of survival functions as shown in Section 3.2. The differences in the inferential results for the gastric cancer dataset and the bone marrow transplant dataset demonstrate the statistical efficiency of our RMST-based Wald test over the existing RMST-based versatile test and log-rank test when the survival functions of two groups cross, which is consistent with the numerical results in Section 3.2.

5. Conclusions

In comparative clinical trials with time-to-event data, the log-rank test is commonly used to examine whether two survival functions are equal or not. Equivalent to the score test for the hypothesis that the hazard ratio is $1$ , the log-rank test is the most powerful test under the PH assumption, which, however, may not hold in practice. To develop a robust omnibus test for the null hypothesis in (1) with comparable power no matter whether the PH assumption is violated or not, we test the null in (1) by comparing a vector of RMSTs between different groups. Interpreting the vector of RMSTs at multiple time points as a statistical functional of the joint survival function of event and censoring times, we define the influence function of RMSTs. Based on the relationship between the influence function and the asymptotic distribution, we estimate the asymptotic covariance matrix of estimated RMSTs via a perturbation procedure and thus construct an RMST-based Wald test for (1), where the statistical inference can be made without intensive numerical resampling schemes (e.g. bootstrap or permutation methods). Extensive numerical studies and real data analysis are conducted to validate the size of the proposed RMST-based Wald test and compare its power with existing methods under different scenarios. It is shown that the proposed RMST-based Wald test with multiple time points possesses comparable power when the PH assumption holds and dominates the log-rank test if the PH assumption is violated. Compared with the RMST-based versatile test ²² which is more similar to the confidence band test, the power of our RMST-based Wald test is more stable across various scenarios, suggesting its statistical robustness.

The RMST-based Wald test can also be extended to the null

$$H_0:S_1=\cdots =S_K$$ (17)

of $K>2$ groups. As the test statistic $T_{\mathit{\psi }}$ in (15) is based on the asymptotic distribution (10), under (17) we have for any $\mathbf{\text{w}}=(w_1,\dots ,w_K{)}^{\mathsf{T}}$ satisfying $\sum _{k=1}^K{w}_k=0$ and $\sum _{k=1}^K{w}_k^2=1$ ,

$$\sum _{k=1}^K{w}_k\mathit{\psi }({\hat{H}}_k)\stackrel{D}{⟶}\text{MVN}(\mathbf{0},\sum _{k=1}^K\frac{w_k^2{\mathbb{V}}_{\mathit{\psi },k}}{n_k})$$

as $min\{n_1,\dots ,n_K\}\to \mathrm{\infty }$ and $n_k/(n_1+\cdots +n_K)\to {\kappa }_k\in (0,1)$ for all $k\in \{1,\dots ,K\}$ . As a result, it is possible to extend the RMST-based Wald test with the test statistic

$$T_{\mathit{\psi },\mathbf{\text{w}}}=\{\sum _{k=1}^K{w}_k\mathit{\psi }({\hat{H}}_k){\}}^{\mathsf{T}}(\sum _{k=1}^K\frac{w_k^2{\mathbb{V}}_{\mathit{\psi },k}}{n_k}{)}^{-1}\{\sum _{k=1}^K{w}_k\mathit{\psi }({\hat{H}}_k)\}$$

which follows the ${\chi }_d^2$ distribution asymptotically when (17) is true. Here, the projection vector $\mathbf{\text{w}}$ controls the power of test with $T_{\mathit{\psi },\mathbf{\text{w}}}$ and the optimal $\mathbf{\text{w}}$ can be estimated following the work of Liu et al. ³⁷

By the definition of RMST in (2), it is clear that there exists a one-to-one correspondence between the survival function $S(t)$ and the RMST $\mu (\tau ;S)$ that

$$\begin{array}{r}S(t)=\frac{\mathrm{\partial }\mu (\tau ;S)}{\mathrm{\partial }\tau }{|}_{\tau =t}\end{array}$$

As a result, the RMST $\mu (\cdot ;S)$ can also be treated as an infinite-dimensional statistical functional of the joint survival function. As presented in Zhao et al., ¹⁸ the process

$$Z(\tau )=\sqrt{n}\{\mu (\tau ;\hat{S})-\mu (\tau ;S)\},\tau \in [0,{\tau }_{\text{max}}]$$

converges weakly to a zero-mean Gaussian process. It is of interest to develop a test based on the process $Z(\tau )$ as an infinite-dimensional extension of our RMST-based Wald test. However, there are several challenges.

• -Given that the covariance function of the process $Z(\tau )$ is an integral of the covariance function (2.2) in the work of Hall and Wellner, ³⁹ the connection between $Z(\tau )$ and standard stochastic processes (Brownian motion or Brownian bridge) is unclear. Existing asymptotic results of standard stochastic processes cannot be directly utilized.
• -As illustrated in Section 2.4, the asymptotic ${\chi }_d^2$ distribution of $T_{\mathit{\psi }}$ under $H_0$ requires the number of time points ( $d$ ) to be fixed and finite. Thus, our proposed inference procedure based on ${\chi }_d^2$ is not directly applicable to developing a test based on $Z(\tau )$ with infinite dimensions.
• -Although resampling approaches are able to provide the exact type I error rate control, directly applying them to develop a test based on $Z(\tau )$ may suffer substantial power loss. For example, similar to construction of the confidence band for the KM estimator,^38–41 we construct a naive band-based test statistic $T_{\mathit{\psi }}$ based on the procedure detailed in Sections 2.2–2.3, where ${\tau }_1,\dots ,{\tau }_d$ are unique values of all the observed event times $\{X_{ki}:{\mathrm{\Delta }}_{ki}=1,k=1,2\}$ . We apply a permutation test (by randomly reassigning observations into two groups) on this band-based test statistic under the settings of Section 2.4 (sample size $n=200$ for both groups, the perturbation parameter $ϵ=0.001$ , significance level $\alpha =0.05$ ). The average value of $d$ over 1000 repetitions is around 220. With the type I error rate ( $5.5\mathrm{\%}$ ) well controlled in the null case (i), the power of the band-based test ( $13.0\mathrm{\%}$ ) is significantly lower than the power of our Wald test ( $38.1\mathrm{\%}$ ) with a fixed number of time points ( $d=4$ ) under the non-null case (c) with $t_0=1$ .

As a result, the proposed construction mechanism of the Wald test statistic cannot be directly utilized for the infinite-dimensional process $\{\mu (\tau ;\hat{S})-\mu (\tau ;S)\}$ and the construction of a powerful band-based test based on the RMST function $\mu (\cdot ;S)$ warrants further investigation.