Bias-adjusted Kaplan-Meier Survival Curves for Marginal Treatment Effect in Observational Studies

Summary:

For time-to-event outcomes, the Kaplan-Meier estimator is commonly used to estimate survival functions of treatment groups and to compute marginal treatment effects, such as the difference in survival rates between treatments at a landmark time. The derived estimates of marginal treatment effect are uniformly consistent under general conditions when data are from randomized clinical trials. For data from observational studies, however, these statistical quantities are often biased due to treatment-selection bias. Propensity score-based methods estimate the survival function by adjusting for the disparity of propensity scores between treatment groups. Unfortunately, mis-specification of the regression model can lead to biased estimates. Using an empirical likelihood (EL) method in which the moments of the covariate distribution of treatment groups are constrained to equality, we obtain consistent estimates of the survival functions and the marginal treatment effect. Equating moments of the covariate distribution between treatment groups simulates the covariate distribution that would have been obtained if the patients had been randomized to these treatment groups. We establish the consistency and the asymptotic limiting distribution of the proposed EL estimators. We demonstrate that the proposed estimator is robust to model mis-specification. Simulation is used to study the finite sample properties of the proposed estimator. The proposed estimator is applied to a lung cancer observational study to compare two surgical procedures in treating early stage lung cancer patients.

1. Introduction

Kaplan-Meier Curves and Measures of Treatment Effect

In cancer and other diseases, time to an event, such as time to death or time to treatment failure, is often used as the clinical outcome for measuring treatment effect. For example, in an observational CALGB lung cancer study (Nwogu et al., 2015), video-assisted thoracic surgery (VATS) and open lobectomy (Open) are two surgical procedures to be compared for their efficacy in treating early stage non-small cell lung cancer patients. The hazard ratio, often estimated via a Cox proportional hazards model, provides a relative measure of treatment effect. Absolute measures of treatment effect, including the difference in survival rates, are useful for evaluating treatment effect when the proportional hazard assumption is violated. Long-term survival effects as in the CALGB VATS vs. Open study and delayed treatment effects in immunotherapy trials are among the reasons that non-proportional hazards exist between treatment regimens in cancer research (Putter et al., 2005; Hoos et al., 2010; Chen, 2013). In these cases, the hazard ratio is no longer a suitable measure for treatment effect and an alternative measure is the difference in survival rates at a landmark time. In this article, we focus on using the Kaplan-Meier estimator (Kaplan and Meier, 1958) to estimate the overall survivals of two or more treatments and compare the difference in survival probabilities at a landmark time. With the CALGB VATS vs. Open study data, we estimate the Kaplan-Meier curves of VATS and Open and compare the survival probabilities of the two surgical procedures at a fixed landmark time.

As is often the case, a randomized clinical trial is not a viable option for comparing treatment regimens due to the high cost in time and resources or to the lack of equipoise for randomization. Real world evidence based on health care records and diseases registries, such as the CALGB VATS vs. Open study, is increasingly used to assess the safety and effectiveness of drugs and devices (Sherman et al., 2016). In the VATS vs. Open example, there are no randomized clinical trials comparing VATS with Open. The CALGB study was designed as a prospective registry trial to evaluate the relative efficacy of the two surgical procedures and standard Kaplan-Meier estimators were used to characterize the survival rates over time (Nwogu et al., 2015). Unfortunately, the Kaplan-Meier estimator based on observational data can be misleading due to unbalanced distribution of baseline covariates between treatment groups. For the VATS vs. Open comparison, it is known that surgeons tend to treat patients with smaller tumors with VATS and patients with larger tumors or more complicated medical conditions with Open. The survival benefit seen in the Kaplan-Meier estimates for VATS relative to Open seen in this study may well be attributed to healthier patients receiving VATS.

Inverse Probability of Treatment Weighting (IPTW) Estimators for Kaplan-Meier Curve

When estimating treatment effects with data from observational studies, it is important to use statistical methods to remove or adjust the effect of treatment-selection bias. Propensity score methods are among the most commonly used statistical methods to minimize the effect of this bias (Austin (2007), Austin et al. (2007), Austin (2011), Deb et al. (2016), Yao et al. (2017)). Once the propensity score is estimated, several methods, including matching on the propensity score (Rosenbaum and Rubin, 1985), stratification on the propensity score (Rosenbaum and Rubin, 1984), and inverse probability of treatment weighting (IPTW) using the propensity score (Rosenbaum, 1987), can be used to remove the confounding effects. Extensive reviews of these methods can be found in D’Agostino (1998) and Lunceford and Davidian (2004).

For survival endpoints, Xie and Liu (2005) proposed an adjusted Kaplan-Meier estimator of the survival function by incorporating IPTWs based on propensity score. Cole and Hernán (2004) proposed a similar inverse weighting of the survival function based on propensity score with a slightly different weight function. Although these methods are relatively easy to implement, several practical difficulties are associated with IPTW methods.

One obvious problem associated with IPTW is that a propensity score close to zero will give an extremely large weight leading to numerical instability and an inflated variance estimate. Another important but often ignored problem for these methods is that the propensity score must be correctly estimated. Defining a propensity score model is difficult, especially when the association of the covariates with treatment selection is not a linear function and involves higher order terms or interactions. It has been found that slight mis-specification of the propensity score model can result in substantial bias of the estimated treatment effects (e.g. Smith and Todd (2005); Kang and Schafer (2007)). The papers from Austin et al. (2007) and Imai and Ratkovic (2014) also demonstrate the importance of correct specification of the propensity score model.

For these reasons, it is important to develop a robust method for bias correction in estimating treatment effect, one not requiring correct specification of the propensity score model. Hirano et al. (2003) developed nonparametric propensity score estimators to provide valid inference on average treatment effect without relying on parametric assumptions in propensity score estimation. These methods require sieve approximations of unknown conditional functions, as these functions appear explicitly in the semiparametric efficient inference functions (Robins et al., 1994). Asymptotically, the estimators converge to the true propensity score function as the sample size increases, but no finite sample properties of these estimators are known and the implementation of the approach is computationally very difficult. Many recent methods focus on improving covariate balance within propensity score and outcome regression frameworks (Qin and Zhang, 2007; Chan et al., 2012; Han and Wang, 2013; Imai and Ratkovic, 2014). In principle, these methods could be extended to survival data in the CALGB VATS vs. Open study to estimate the survival functions and the absolute marginal treatment effect, but they all require parametric or nonparametric modeling of the propensity score or the outcome model and thus encounter similar theoretical and computational difficulties.

The Proposed Method

It is natural to question whether estimation of these propensity score functions is even necessary, and whether they can be replaced by directly balancing the distribution of baseline covariates.

The goal of this paper is to study an empirical likelihood-based method (e.g. Owen (2001); Qin (2017)) to enforce the equality of the covariate distributions, more specifically the equality of the moments of the covariates between treatment groups. After obtaining the empirical probability mass for each observation, we are able to estimate consistently the survival function for the treatment group and the control group as well as the marginal treatment effects between treatment groups. A Wald-type statistics can be used to test the absolute difference in survival rates between treatment groups at a pre-specified landmark time. The resulting estimator is a nonparametric estimator for the marginal survival function. We also propose variance estimators for these statistical quantities.

The obvious advantage of the proposed approach is that there is no need to estimate a propensity score in estimating marginal treatment effect. The inverse weighting methods used to estimate the Kaplan-Meier curves require specification of either a propensity score model, an outcome model or both. Consistency of the estimators requires some underlying models to be correctly specified. Since the parameter of interest is the average treatment effect based on the Kaplan-Meier estimates, it is more natural and robust to directly balance the covariate distribution and to bypass the estimation of either the propensity score or the outcome models. The approach of balancing the covariate distribution between treatments has been discussed (e.g. Hellerstein and Imbens (1999); Hainmueller (2011); Chan et al. (2012, 2016)). In particular, Chan et al. (2016) proposed estimation of average treatment effects by empirical balancing calibration weights and demonstrated that the estimator can achieve global efficiency, which means the estimator is able to achieve semiparametric efficiency bounds without the requirement of correct specification of propensity score or outcome regression models. This method is related to ours, but the discussion is generic and does not cover the estimation of survival functions and absolute treatment effect based on survival functions.

The paper will be organized as follows. Section 2 provides notation and a description of the proposed empirical likelihood (EL) based method for estimating marginal survival functions. Section 3 establishes the large sample properties of the estimators for marginal survival function and the absolute difference in survival rates at time $t$. In Section 4, the finite sample properties of the proposed estimators, relative to the IPTW estimator and the standard Kaplan-Meier estimators are studied via simulations. In Section 5, the proposed method is illustrated using the CALGB lung cancer registry data to compare survival function between VATS and Open. The paper concludes with discussion of various issues in Section 6. The details of the asymptotic properties of the proposed estimators and their proofs are given in the online Supplementary Material.

2. Notation and the Proposed Estimator

Let $Z$ be a $l\times 1$ vector of covariates, and $D$ be an indicator of observed treatment status, where $D=1$ for the treatment group, and $D=0$ for the control group. Let $T_1$ be the survival time if a patient receives the treatment and $T_0$ be the survival time if a patient receives the control. The corresponding survival functions are denoted by $S_1(t)$ and $S_0(t)$ respectively. Note that, $(T_1\mathrm{,}{T}_0)$ cannot be observed simultaneously for an individual. For any individual in the treatment group, only $T_1$ can be observed; for any individual in the control group, only $T_0$ can be observed. Let $C$ be the censoring time. Throughout this paper, independent censorship is assumed, i.e., $C$ is independent of $(T_1\mathrm{,}{T}_0)$. To estimate the difference between the two groups, strongly ignorable treatment assignment $(T_1\mathrm{,}{T}_0)\perp \text{l}D|Z$ is assumed. The parameter of interest is the difference of the survival rates at time $t$, which is defined by $\mathrm{\Delta }(t)=S_1(t)-S_0(t)$.

In real world evidence studies, the covariate distribution between treatment groups are often unbalanced, leading to treatment-selection bias. Consequently, the standard Kaplan-Meier estimate of the survival function is biased in this situation. In order to reduce this selection bias, a new estimator for survival rate based on an empirical likelihood is proposed. For simplicity, we only discuss the estimation of $S_1(t)$; a similar derivation holds for $S_0(t)$.

Denote $T=D{T}_1+(1-D)T_0\mathrm{.}$ Furthermore, let $Y_1=T_1\wedge C\mathrm{,}{\delta }_1=I(T_1\le C)$ and $Y_0=T_0\wedge C\mathrm{,}{\delta }_0=I(T_0\le C)$. If $D_i=\mathrm{1,}$ the observed data are $(Z_i\mathrm{,}{Y}_{1i}\mathrm{,}{\delta }_{1i});$ if $D_i=\mathrm{0,}$ the observed data are $(Z_i\mathrm{,}{Y}_{0i}\mathrm{,}{\delta }_{0i})$.

Let $n_1={\sum }_{i=1}^n{D}_i$ and $n_0={\sum }_{i=1}^n(1-D_i)$. For simplicity, denote the $n_1$ observations with $D_i=1$ as $(Z_i\mathrm{,}{Y}_{1i}\mathrm{,}{\delta }_{1i})\mathrm{,}$ $i=\mathrm{1,}\cdots \mathrm{,}{n}_1;$ the other $n_0$ observations with $D_i=0$ as $(Z_i\mathrm{,}{Y}_{0i}\mathrm{,}{\delta }_{0i})\mathrm{,}$ $i=n_1+\mathrm{1,}\cdots \mathrm{,}n\mathrm{.}$ Let ${\overline{N}}_1(u)={\sum }_{i=1}^{n_1}I(Y_{1i}\le u\mathrm{,}{\delta }_{1i}=1)$ and ${\overline{Y}}_1(u)={\sum }_{i=1}^{n_1}I(Y_{1i}\ge u)$, then the Kaplan-Meier estimator of $S_1(t)$ (Kaplan and Meier, 1958) is defined as:

$${\widehat{S}}_1^{km}(t)=\prod _{u\le t}\{1-\frac{d{\overline{N}}_1(u)}{{\overline{Y}}_1(u)}\}\mathrm{.}$$

However, ${\widehat{S}}_1^{km}(t)$ is biased when there exists treatment-selection bias in the data.

We use an empirical likelihood approach to address treatment-selection bias. If $D_i=\mathrm{1,}$ the observed data are $(Z_i\mathrm{,}{Y}_{1i}\mathrm{,}{\delta }_{1i})\mathrm{,}$ and the density function is denoted by $q_i\mathrm{.}$ If $D_i=\mathrm{0,}$ the observed data are $(Z_i\mathrm{,}{Y}_{0i}\mathrm{,}{\delta }_{0i})$, and the density function is denoted by $r_i\mathrm{.}$ Let $p_i=q_i^{D_i}{r}_i^{(1-D_i)}\mathrm{,}$ $i=\mathrm{1,}\cdots \mathrm{,}n\mathrm{.}$ For the observed data, the likelihood function is

$$\mathcal{L}=\prod _{i=1}^n{p}_i\mathrm{.}$$

Since the covariates $Z$ are always observed for both the treatment group and the control group, we can estimate $p_i$ by maximizing the likelihood function subject to the following constraints:

$$p_i\ge \mathrm{0,}\sum _{i=1}^n{p}_i{D}_i=\mathrm{1,}\sum _{i=1}^n{p}_i(1-D_i)=\mathrm{1,}$$ (2.1)

$$\sum _{i=1}^n{p}_i{D}_i\varphi (z_i)=\sum _{i=1}^n{p}_i(1-D_i)\varphi (z_i)\mathrm{,}$$ (2.2)

where $\varphi (z)={(\varphi (z_1)\mathrm{,}\varphi (z_2)\mathrm{,}\cdots \mathrm{,}\varphi (z_m))}^T$ is a $m\times 1$ vector of independent known functions of the covariates $z$ and is bounded.

The first constraint (2.1) reflects the fact that both $q_i$ and $r_i$ are density functions. The second constraint (2.2) enforces the equality of the moments of the covariates between treatment groups. Let $a=E\varphi (z)\mathrm{.}$ Using the Lagrange multiplier algorithm, maximizing $\mathcal{L}$ subjecting to the constraints (2.1) and (2.2), we obtain

$$p_i{D}_i=\frac{D_i}{n_1+{\gamma }^T(\varphi (z_i)-a)}\mathrm{,}$$ (2.3)

$$p_i(1-D_i)=\frac{1-D_i}{n_0-{\gamma }^T(\varphi (z_i)-a)}\mathrm{,} i=\mathrm{1,}\cdots \mathrm{,}n\mathrm{,}$$ (2.4)

where $\gamma$ and $a$ satisfy the following estimating equations:

$$\sum _{i=1}^n\frac{D_i(\varphi (z_i)-a)}{n_1+{\gamma }^T(\varphi (z_i)-a)}=0$$ (2.5)

$$\sum _{i=1}^n\frac{(1-D_i)(\varphi (z_i)-a)}{n_0-{\gamma }^T(\varphi (z_i)-a)}=0.$$ (2.6)

Then a new estimator of the survival function $S_1(t)$, an empirical likelihood-based (EL-based) estimator ${\widehat{S}}_1(t)$ is:

$${\widehat{S}}_1(t)=\prod _{u\le t}\{1-\frac{d{\overline{N}}_1^{EL}(u)}{{\overline{Y}}_1^{EL}(u)}\}\mathrm{,}$$ (2.7)

where ${\overline{N}}_1^{EL}(u)={\sum }_{i=1}^n{p}_i{D}_{i}I(Y_{1i}\le u\mathrm{,}{\delta }_{1i}=1)$ and ${\overline{Y}}_1^{EL}(u)={\sum }_{i=1}^n{p}_i{D}_{i}I(Y_{1i}\ge u)$. The estimator ${\widehat{S}}_1(t)$ removes the selection bias.

The usual choice for $\varphi (Z)$ is $\varphi (Z)=Z$ or $\varphi (Z)=(Z\mathrm{,} Z{Z}^T)\mathrm{,}$ i.e., adjusting the first moment or the first and second moment of the covariates. For one dimensional variable $Z$, $\varphi (Z)=Z$ forces mean equality between treatment groups and $\varphi (Z)=(Z\mathrm{,}{Z}^2\mathrm{,}\cdots \mathrm{,}{Z}^k)(k\ge 2)$ forces the first $k$-th moment equality between treatment groups. An explanation of why treatment-selection bias can be removed by this approach may be seen in the following. Common distributions are determined completely by a finite number of parameters. The parameters usually can be estimated by moment estimation methods and are often consistent. In our approach, the constraint (2.2) with $\varphi (Z)=(Z\mathrm{,}{Z}^2\mathrm{,}\cdots \mathrm{,}{Z}^k)$ implies that the moments of the covariates between treatment groups are the same. This will lead to equality of the distribution of the covariates between treatment groups. Thus, treatment-selection bias can be removed. A lower order $k$ often is enough to remove most of the selection bias. For example, adjusting on the first moments of the covariates and the second moment of the joint distribution of all paired covariates is often sufficient in most applications.

In the next section, we discuss the large sample properties of ${\widehat{S}}_1(t)$ including consistency and the limiting distribution.

3. Large Sample Properties for ${\widehat{S}}_k(t)$ and $\widehat{\mathrm{\Delta }}(t)$

Under the assumptions in our paper, there exists a constant $c\in (\mathrm{0,1})$, such that as $n\to \infty \mathrm{,}$ $n_1/n\to c\mathrm{.}$ Throughout the remainder of the paper, we let $||\cdot ||$ denote Euclidean norm and $\psi (Z)={(\mathrm{1,}{\varphi }^T(Z))}^T\mathrm{.}$ Furthermore, for any matrix $B$, $B^T$ denotes the transpose of $B$ and $\stackrel{\mathcal{D}}{\to }$ denotes “converges in distribution.”

Propensity score $e(Z)=P(D=1|Z)$ describes the probability of treatment given the covariates $Z$. As in Rosenbaum and Rubin (1983), we assume that the propensity score is bounded away from 0 and 1, that is, there exists $\varsigma >\mathrm{0,}$ such that $\varsigma <P(D=1|Z)<1-\varsigma$ for any $Z$. We assume that the propensity score $e(Z)$ is contained in the linear space spanned by $\psi (Z)\mathrm{.}$ Note that this condition is used solely to derive the large sample properties and the proposed estimators do not rely on the value of the estimated propensity score. This condition precludes lengthy technical arguments and theoretical discussions. All the proofs of the theoretical results are given in the online Supplementary Material.

Theorem 1: (Consistency theorem) $T_1$ is a failure time random variable with continuous survival function $S_1(s)$, and the distribution function is $F_1(s)=1-S_1(s)\mathrm{.}$ The cumulative hazard function is ${\mathrm{\Lambda }}_1(s)$ and ${\mathrm{\Lambda }}_1(s)={\int }_0^{s}d{F}_1(v)/S_1(v)\mathrm{.}$ Let ${\overline{Y}}_1(s)={\sum }_{i=1}^{n_1}I(Y_{1i}\ge s)$ and $t\in (\mathrm{0,}\infty ]$ such that as $n_1\to \infty \mathrm{,}$ ${\overline{Y}}_1(t)\stackrel{P}{\to }\infty \mathrm{.}$ Assume that $E[\psi (Z){\psi }^T(Z)]$ is positive definite and $E||\psi (Z){||}^3<\infty \mathrm{.}$ The EL-based estimator ${\widehat{S}}_1(t)$ is defined in (2.7), is an estimator of $S_1(t)\mathrm{.}$ Then

$$\underset{0\le s\le t}{\sup }|{\widehat{S}}_1(s)-S_1(s)|\stackrel{P}{\to }\mathrm{0,}\hspace{1em}as\hspace{1em}n\to \infty \mathrm{.}$$

Theorem 2: (Limiting theorem) Denote ${\pi }_1(t)=P(Y_1\ge t)$ and ${\mathcal{I}}_1=\{t :\hspace{0.17em}{\pi }_1(t)>0\}$. Assume that $E[\psi (Z){\psi }^T(Z)]$ is positive definite and $E||\psi (Z)|{|}^3<\infty \mathrm{.}$ Assume as $n_1\to \infty \mathrm{,}$ ${\sup }_{0\le t<\infty }\left|\frac{{\overline{Y}}_1(t)}{n_1}-{\pi }_1(t)\right|\stackrel{P}{\to }\mathrm{0,}$ and $F_1(\cdot )$ is continuous. Then, as $n\to \infty \mathrm{,}$ for any $t\in {\mathcal{I}}_1\mathrm{,}$

$$\sqrt{n}\left(\frac{{\widehat{S}}_1(t)-S_1(t)}{S_1(t)}\right)\stackrel{\mathcal{D}}{\to }N(\mathrm{0,}{V}_1)\mathrm{,}$$ (3.1)

where

$$V_1=E{\left[{\int }_0^t\frac{1}{{\pi }_1(s)}\frac{D}{e(Z)}d{M}_1(s)\right]}^2-B_1{A}^{-1}{B}_1^T$$

and $A=E\left[\frac{\psi (Z){\psi }^T(Z)}{e(Z)(1-e(Z))}\right]$, $B_1=E\left[\frac{D{\psi }^T(Z)}{e^2(Z)}{\int }_0^t\frac{1}{{\pi }_1(s)}d{M}_1(s)\right]$, $M_1(s)=I(Y_1\le s\mathrm{,}{\delta }_1=1)-{\int }_0^{t}I(Y_1\ge s)d{\mathrm{\Lambda }}_1(s)\mathrm{.}$

Since $V_1$ involves the propensity score $e(Z)$ and, in general, we do not know the true propensity score model, a nonparametric bootstrap method is recommended to estimate the variance. If we know the true model for $e(Z)\mathrm{,}$ then by fitting the model based on the data, an estimate of $e(Z)$ can be obtained, and this estimate can be plugged into $V_1$ to obtain the estimated variance of ${\widehat{S}}_1(t)$.

We can obtain the large sample properties for the estimate of $S_0(t)$ and the difference $\mathrm{\Delta }(t)$ in the same way, where the estimates are defined as :

$${\widehat{S}}_0(t)=\prod _{u\le t}\{1-\frac{{\sum }_{i=1}^n{p}_i(1-D_i)I(Y_{0i}\le u\mathrm{,}{\delta }_{0i}=1)}{{\sum }_{i=1}^n{p}_i(1-D_i)I(Y_{0i}\ge u)}\}\mathrm{,}$$

and

$$\widehat{\mathrm{\Delta }}(t)={\widehat{S}}_1(t)-{\widehat{S}}_0(t)\mathrm{.}$$

We can obtain the large sample results for ${\widehat{S}}_0(t)$. The derivations are similar and we omit them here.

Theorem 3: (Limiting theorem) Denote ${\pi }_0(t)=P(Y_0\ge t)$ and $\mathcal{I}=\{t :\hspace{0.17em}{\pi }_0(t)>0\}$. Assume that $E[\psi (Z){\psi }^T(Z)]$ is positive definite and $E||\psi (Z){||}^3<\infty \mathrm{.}$ Assume as $n_0\to \infty \mathrm{,}$ ${\sup }_{0\le t<\infty }\left|\frac{{\overline{Y}}_0(t)}{n_0}-{\pi }_0(t)\right|\stackrel{P}{\to }\mathrm{0,}$ and $S_0(\cdot )$ is continuous. The cumulative hazard function of $T_0$ is ${\mathrm{\Lambda }}_0(s)$. Then, as $n\to \infty \mathrm{,}$ for any $t\in \mathcal{I}\mathrm{,}$

$$\sqrt{n}\left(\frac{{\widehat{S}}_0(t)-S_0(t)}{S_0(t)}\right)\stackrel{\mathcal{D}}{\to }N(\mathrm{0,}{V}_0)\mathrm{,}$$

where

$$V_0=E{\left[\frac{1-D}{1-e(Z)}{\int }_0^t\frac{1}{{\pi }_0(s)}d{M}_0(s)\right]}^2-B_0{A}^{-1}{B}_0^T$$

and $A=E\left[\frac{\psi (Z){\psi }^T(Z)}{e(Z)(1-e(Z))}\right]$, $B_0=E\left[\frac{(1-D){\psi }^T(Z)}{{(1-e(Z))}^2}{\int }_0^t\frac{1}{{\pi }_0(s)}d{M}_0(s)\right]$, $M_0(s)=I(Y_0\le s\mathrm{,}{\delta }_0=1)-{\int }_0^{t}I(Y_0\ge s)d{\mathrm{\Lambda }}_0(s)\mathrm{.}$

Since ${\widehat{S}}_1(t)$ and ${\widehat{S}}_0(t)$ are correlated by some common covariates, the large sample property of $\widehat{\mathrm{\Delta }}(t)={\widehat{S}}_1(t)-{\widehat{S}}_0(t)$ is complicated. Its large sample properties are given in the following theorem.

Theorem 4: (Limiting theorem)${\pi }_1(t)$, ${\pi }_0(t)$, ${\mathrm{\Lambda }}_1(t)$ and ${\mathrm{\Lambda }}_0(t)$ are defined as before. $\mathcal{I}=\{t :\hspace{0.17em}{\pi }_1(t){\pi }_0(t)>0\}$. Assume that $E[\psi (Z){\psi }^T(Z)]$ is positive definite and $E||\psi (Z)|{|}^3<\infty \mathrm{.}$ Assume as $n\to \infty \mathrm{,}$${\sup }_{0\le t<\infty }\left|\frac{{\overline{Y}}_0(t)}{n_0}-{\pi }_0(t)\right|\stackrel{P}{\to }0$, ${\sup }_{0\le t<\infty }\left|\frac{{\overline{Y}}_1(t)}{n_1}-{\pi }_1(t)\right|\stackrel{P}{\to }0$. Both $S_1(\cdot )$ and $S_0(t)$ are continuous. Then, as $n\to \infty \mathrm{,}$ for any $t\in \mathcal{I}\mathrm{,}$

$$\sqrt{n}\left(\widehat{\mathrm{\Delta }}(t)-\mathrm{\Delta }(t)\right)\stackrel{\mathcal{D}}{\to }N(\mathrm{0,}V)\mathrm{,}$$

where $B=S_1(t)B_1+S_0(t)B_0$, $V_1\mathrm{,} V_0\mathrm{,} B_{\mathrm{1,}} B_0$ are defined in Theorem 2 and Theorem 3 and

$$V=S_1^2(t)V_1+S_0^2(t)V_0+B{A}^{-1}{B}^T-2S_1(t)B{A}^{-1}{B}_1^T-2S_0(t)B{A}^{-1}{B}_0^T\mathrm{.}$$

4. Simulation Studies

To illustrate the performance of the proposed EL method, we compare it with two existing methods: the inverse probability of treatment weighting (IPTW) by Xie and Liu (2005) and the standard Kaplan-Meier method (KM). Our first simulation study is an example with a large selection bias and the second is one with a moderate selection bias.

Simulation study 1

First, we study a situation with a large selection bias. Consider covariates $Z=(Z_1\mathrm{,}{Z}_2{)}^T\mathrm{,}$ where $Z_1\sim U(0,1)$, a uniform distribution on $[0,1]$ and $Z_2$ follows a exponential distribution with mean $1$. The true propensity score $e(Z)$ is

$$P(D=1|Z)=\frac{\exp (0.8\exp (Z_1)-1.4Z_2^{1.1})}{1+\exp (0.8\exp (Z_1)-1.4Z_2^{1.1})}\mathrm{.}$$

In this model, the treatment variable is strongly related to the covariates and a large selection bias exists. For the treatment group, the hazard function of $T_1$ is ${\lambda }_1(t|Z)={\lambda }_{10}(t)\exp (Z^{{*}^T}{\beta }_1)\mathrm{,}$ where $Z^{*}=(Z_1{Z}_2\mathrm{,}{Z}_2{)}^T$ and ${\lambda }_{10}(t)=t\exp (-2)\mathrm{.}$ For the control group, the hazard function of $T_0$ is ${\lambda }_0(t|Z)={\lambda }_{00}(t)\exp (Z^T{\beta }_0)\mathrm{,}$ where $Z=(Z_1\mathrm{,}{Z}_2{)}^T$ and ${\lambda }_{00}(t)=t\exp (-2)\mathrm{.}$ The observed survival time is $T=D{T}_1+(1-D)T_0\mathrm{.}$ The censoring variable $C$ follows a uniform distribution on $[\mathrm{0,}c]\mathrm{.}$ The true value of parameters are ${\beta }_1={(3,1)}^T\mathrm{,}$ ${\beta }_0={(2,3)}^T$ and $c=\mathrm{6.6.}$

For a fixed point $t^{*}=1.18$, we estimate the survival rates for the treatment group and the control group respectively and the difference in survival rates at $t^{*}$. Note that the choice of $t^{*}=1.18$ is arbitrary; the choice of a different time point would produce similar results. The IPTW method requires specification of a model for the propensity score $e(Z)$. Since the true model is unknown, there is a possibility that it is misspecified. In the simulation, we incorrectly model the propensity score $e(Z)=P(D=1|Z)$ directly on the covariates $Z$ rather than $Z^{*}$ by a logistic regression model. In our simulation, the sample size is 600 with $n_0=n_1=300$. The simulation results given in Table 1 are based on 1000 simulations.

Table 1.

Results of simulation study 1 (large selection bias)

method	para	true	est.hat	bias	se	sd	RMSE
KM	$S_1(t)$	0.504	0.626	0.242	0.03	0.029	0.059
KM	$S_0(t)$	0.265	0.169	0.362	0.023	0.023	0.132
KM	$\mathrm{\Delta }(t)$	0.239	0.457	0.912	0.038	0.037	0.833
IPTW Correct	$S_1(t)$	0.504	0.52	0.032	0.036	0.032	0.002
IPTW Correct	$S_0(t)$	0.265	0.267	0.007	0.034	0.036	0.001
IPTW Correct	$\mathrm{\Delta }(t)$	0.239	0.253	0.06	0.047	0.048	0.006
IPTW Incorrect	$S_1(t)$	0.504	0.557	0.104	0.036	0.033	0.012
IPTW Incorrect	$S_0(t)$	0.265	0.234	0.116	0.032	0.034	0.014
IPTW Incorrect	$\mathrm{\Delta }(t)$	0.239	0.322	0.349	0.047	0.048	0.124
EL adjust for $Z$	$S_1(t)$	0.504	0.502	0.003	0.037	0.035	0.001
EL adjust for $Z$	$S_0(t)$	0.265	0.268	0.013	0.039	0.034	0.002
EL adjust for $Z$	$\mathrm{\Delta }(t)$	0.239	0.234	0.021	0.05	0.05	0.003

bias: the relative bias; true: the true value of the parameters; se: the standard deviation of the estimators in the simulation; sd: the mean of the standard deviation of the estimators; RMSE is the root means square error; IPTW Correct: refers to the IPTW estimator when the propensity score $e(Z)$ is correctly specified; IPTW Incorrect: refers to the IPTW estimator when the propensity score $e(Z)$ is misspecified.

In the simulation tables, bias is the relative bias, true means the true value of the parameters, se is the standard deviation of the estimators in the simulation, sd is the mean of the standard deviation of the estimators, and RMSE is the root means square error. “IPTW Correct” refers to the IPTW estimator when the propensity score $e(Z)$ is correctly specified and “IPTW Incorrect” refers to the IPTW estimator when the propensity score $e(Z)$ is misspecified.

From Table 1, we can see that if selection bias is ignored, the standard Kaplan-Meier estimator is very biased; the relative bias of the difference $\mathrm{\Delta }(t)$ is 0.912. When the propensity score model is correct, the IPTW method effectively removes bias and performs well. However, when the propensity score model is incorrect, the IPTW method is biased. For $\varphi (z)=z$, adjusting only the first moment of the covariates, our proposed EL estimator removes completely the bias.

In Figure 1, we give the plots of the entire survival function $S_1(t)\mathrm{,} S_0(t)$ and their difference $\mathrm{\Delta }(t)\mathrm{.}$ The plots are based on 300 samples of size $n=600$. The upper curves represents the curve of the survival function $S_1(t)$ of $T_1\mathrm{,}$ and the lower curves represents the curve of the survival function $S_0(t)$ of $T_0\mathrm{.}$ The difference in survival rates denotes $\mathrm{\Delta }(t)=S_1(t)-S_0(t)\mathrm{.}$ The dot-dash line is obtained by the EL method, the dash line from the standard method, and the solid line represents the true curve. Figures 1 shows that the EL method yields satisfactory estimates of the true survival function and their difference, and they are much better than those from the standard KM estimators.

Figure 1.

Simulation study 1 – estimates of survival functions.

Simulation study 2

We next consider the situation when there is moderate selection bias in the data.

All the settings are identical to those in simulation study 1 except that the true propensity score $e(Z)$ is

$$P(D=1|Z)=\frac{\exp \hspace{0.17em}(0.3\exp (Z_1)-0.5Z_2^{1.1})}{1+\exp \hspace{0.17em}(0.3\exp (Z_1)-0.5Z_2^{1.1})}\mathrm{.}$$

The simulation results are given in Table 2. The number of simulations is 1000. CVP is the $95\%$ empirical coverage. As expected, the standard Kaplan-Meier estimator is biased, reflecting the moderate selection bias in the data. When treatment assignment model is correctly specified, the IPTW estimator performs well. There is always the possibility of model mis-specification, and when the treatment assignment model is misspecified, the IPTW estimator for the difference $\mathrm{\Delta }(t)$ remains biased. Furthermore, the $95\%$ empirical coverage for $S_1(t)$ and $S_0(t)$ is far from the nominal coverage.

Table 2.

Results of simulation study 2 (moderate selection bias)

method	para	true	est.hat	bias	se	sd	RMSE	CVP
KM	$S_1(t)$	0.504	0.566	0.122	0.031	0.03	0.126	43.6
KM	$S_0(t)$	0.265	0.223	0.161	0.025	0.025	0.163	60.3
KM	$\mathrm{\Delta }(t)$	0.239	0.343	0.437	0.04	0.039	0.439	26.1
IPTW correct	$S_1(t)$	0.504	0.507	0.005	0.029	0.03	0.029	95.6
IPTW correct	$S_0(t)$	0.265	0.266	0.003	0.028	0.029	0.028	96
IPTW correct	$\mathrm{\Delta }(t)$	0.239	0.241	0.008	0.038	0.042	0.039	97.5
IPTW incorrect	$S_1(t)$	0.504	0.521	0.034	0.03	0.03	0.045	91.1
IPTW incorrect	$S_0(t)$	0.265	0.254	0.04	0.028	0.029	0.049	93.5
IPTW incorrect	$\mathrm{\Delta }(t)$	0.239	0.266	0.116	0.039	0.041	0.123	91.2
EL adjust for $Z$	$S_1(t)$	0.504	0.504	0	0.027	0.029	0.027	96.5
EL adjust for $Z$	$S_0(t)$	0.265	0.266	0.002	0.028	0.028	0.028	94.9
EL adjust for $Z$	$\mathrm{\Delta }(t)$	0.239	0.238	0.002	0.036	0.038	0.036	95.6
EL adjust for $Z$(boot)	$S_1(t)$	0.504	0.503	0.002	0.027	0.027	0.028	94.4
EL adjust for $Z$(boot)	$S_0(t)$	0.265	0.264	0.003	0.028	0.028	0.028	95.3
EL adjust for $Z$(boot)	$\mathrm{\Delta }(t)$	0.239	0.239	0.001	0.037	0.036	0.037	94.3
EL adjust for $(Z\mathrm{,}Z{Z}^T)$	$S_1(t)$	0.504	0.505	0.003	0.027	0.026	0.028	93.5
EL adjust for $(Z\mathrm{,}Z{Z}^T)$	$S_0(t)$	0.265	0.261	0.015	0.026	0.026	0.03	95.2
EL adjust for $(Z\mathrm{,}Z{Z}^T)$	$\mathrm{\Delta }(t)$	0.239	0.244	0.023	0.033	0.033	0.041	94.0

EL adjust $z$(boot): the variance is estimated by a bootstrapped method; CVP: the 95% empirical coverage

For the EL estimator, we consider two cases. The first case $\varphi (Z)=Z$ corresponds to adjusting only for the first moment of the covariates. The results shows that the proposed EL estimator is unbiased and the sd is not very far from the se. The $95\%$ empirical coverages for the three parameters $S_1(t)$, $S_0(t)$ and $\mathrm{\Delta }(t)$ are very close to the nominal coverage. The second case $\varphi (Z)=(Z\mathrm{,} Z{Z}^T)$ corresponds to adjusting for the first and second moments of the covariates. In this case, the EL estimator is unbiased and is more efficient than that of the first case with smaller standard deviation (sd). These results also demonstrate that adjusting a finite order moment of the covariates is ordinarily sufficient to remove the bias.

In simulation studies, we know the true propensity score function, so for simplicity we estimate $\widehat{e}(z)$ according to the true propensity score model and then plug $\widehat{e}(z)$ into the variance expressions in Section 3 to obtain an estimate of the variance. As noted previously, when the true treatment assignment model is unknown in data analysis, one can use a bootstrap method to estimate the variance. To demonstrate the performance of the bootstrap variance estimator, we give the result for the EL method adjusting for $Z$. With 300 bootstrap replications resampled from the data, the corresponding results are listed in Table 2, denoted as “EL adjust $z$(boot)”. Compared to the simulated standard error, the bootstrap method estimates the variance of the proposed estimator quite well. The results at other time points are similar but not shown here.

As shown in Table 2, there are no substantial differences in the performance of the EL estimator when either the first moments of $Z$ or both the first and the second moments of $Z$ are used. Additional simulations show that the EL estimator is more efficient than the KM estimator when there is no selection bias, and the results hold across a range of of treatment assignment proportions $P(D=1)$, time points $t^{*}$ and sample sizes $n$. This finding suggests that the EL estimator is also an efficient method for covariate adjustment in randomized clinical trials where selection bias does not exist.

5. Data Example

In this section, we re-analyze the observational data extracted from the CALGB lung cancer registry (and originally reported by Nwogu et al. (2015)). There exists selection bias in the data and the proposed EL method is used to compare the two surgical procedures (VATS and Open) in treating non-small cell lung cancer.

The study cohort contains the patients who enrolled between October 2004 and June 2010 from 15 institutions contributing to the CALGB lung cancer registry. There are 519 observations collected from fifteen sites. Rhode Island only performed Open lobectomy (59 patients), all of which were performed by one surgeon. Cedars-Sinai only performed VATS lobectomy (92 patients), 85% of which were performed by one surgeon. Since the two institutions have $e(z)$ very close to $0$ or $1$, we exclude the observations from the two institutions in our analysis.

The observed covariates include age, gender, performance status, tumor size, insurance, race, any medical co-morbidities, pathologic stage, any symptoms and history of smoking and some other variables. The difference in covariates between the two groups is evaluated by a Fisher’s exact test for binary variables and Wilcoxon rank sum test for continuous variables. An initial check for the balance of the covariates in the two groups reveals that the covariates which cause selection bias are tumor size, race, insurance, pathologic stage, history of smoking and any symptoms. The outcome of interest is disease-free survival time, time from the surgery to the initial disease recurrence, a commonly used endpoint in lung cancer studies.

As in the simulation study, we give the results for three estimators: the standard Kaplan-Meier (KM) estimator, the IPTW estimator and the EL estimator. For the IPTW estimator, we model the propensity score $e(Z)$ by a logistic model. An EL method adjusting for all the six covariates is used. The EL estimator is based on the first moment constraint of the covariates. The variance of the EL estimator is estimated by the bootstrap method with 300 bootstraps.

The results for the point estimates at times $(t_1\mathrm{,}{t}_2\mathrm{,}{t}_3)=(36,48,56)$, are given in Table 3, where $est\mathrm{.}{t}_1\mathrm{,}$ $est\mathrm{.}{t}_2\mathrm{,}$ and $est\mathrm{.}{t}_3$ denote the estimator obtained by the three methods and $sd\mathrm{.}{t}_1\mathrm{,}$ $sd\mathrm{.}{t}_2\mathrm{,}$ and $sd\mathrm{.}{t}_3$ denote the standard deviation derived by the bootstrap method. We also give the Z-ratio for the difference in survival rates between VATS and Open, which are denoted as $Z_{t_1}$, $Z_{t_2}$ and $Z_{t_3}$ respectively. At $t_1=\mathrm{36,}$ the difference is significant by the unadjusted Kaplan-Meier estimator and it is non-significant for the IPTW estimator and EL estimator; and at $(t_2\mathrm{,}{t}_3)=(48,56)$, all three estimators are non-significant with a large bias in the Kaplan-Meier estimator. Thus the unadjusted estimator overestimates the survival difference between VATS and Open. The Kaplan-Meier estimator ignoring the bias yields different conclusions from the IPTW method or the EL method. Furthermore, the EL estimator of $\mathrm{\Delta }(t)$ is more efficient than the IPTW estimator.

Table 3.

Lung cancer example: Disease-free survival at $(t_1\mathrm{,}{t}_2\mathrm{,}{t}_3)$

method	para	est.$t_1$	sd.$t_1$	$Z_{t_1}$	est.$t_2$	sd.$t_2$	$Z_{t_2}$	est.$t_3$	sd.$t_3$	$Z_{t_3}$
KM	$S_1(t)$	0.637	0.035		0.543	0.037		0.493	0.039
KM	$S_0(t)$	0.534	0.038		0.468	0.038		0.438	0.039
KM	$\mathrm{\Delta }(t)$	0.104	0.052	2	0.075	0.053	1.42	0.055	0.055	1
IPTW	$S_1(t)$	0.568	0.039		0.483	0.039		0.443	0.04
IPTW	$S_0(t)$	0.577	0.049		0.511	0.051		0.485	0.052
IPTW	$\mathrm{\Delta }(t)$	−0.01	0.063	0.15	−0.028	0.064	0.44	−0.041	0.066	0.63
EL	$S_1(t)$	0.573	0.043		0.489	0.04		0.448	0.041
EL	$S_0(t)$	0.57	0.042		0.503	0.043		0.474	0.043
EL	$\mathrm{\Delta }(t)$	0.003	0.061	0.05	−0.013	0.061	0.22	−0.026	0.056	0.46

Figure 2 gives the estimated survival curves. There appears to be a difference between VATS and Open from the Kaplan-Meier (KM) estimator but no difference based on IPTW estimator and the EL estimator. The difference between VATS and Open by the EL method is smaller than that obtained by the IPTW method.

Figure 2.

Survival function estimates for CALGB lung cancer data.

6. Discussion

In this paper, we propose an empirical likelihood (EL) based estimator that balances the covariate distribution by forcing equality of moments of the covariates. As the case for the Kaplan-Meier estimator, our EL estimator is a nonparametric method. Unlike Cole and Hernán (2004) and Xie and Liu (2005), the EL estimator allows counterfactual estimation on survival functions without the requirement of a correct specification of the propensity score model, an elusive goal with no effective tool to assess the lack of fit. The EL method involves estimating the EL-based weights through solving a system of equations for moment constraints and there exist cases where no valid solutions of these equations can be found. As with other propensity score-based methods, this issue can be mitigated by collecting more data or by ensuring significant data overlap between treatment groups. The EL method mimics randomized clinical trials through the constraints of the moments between treatment groups. The validity of the EL estimator has been proven under the assumption that the propensity score is contained in the linear space spanned by $\psi (Z)$. This assumption simplifies the proofs of the theorems, but the calculation of the EL estimator does not involve an estimation of the propensity score.

In principle, one could prove these theorems without the propensity score assumption, but additional conditions would need to be added to obtain the large sample properties. First, the propensity score could be approximated by a fully nonparametric sieve estimate. Then we could add some smooth conditions on the propensity score and some other nonparametric conditions. Next, based on nonparametric theory, a similar but different statement to this assumption could be obtained. Finally, the large sample properties could be derived. However, for the estimator to be consistent, the number of the constraints would go to infinity. This is impossible in applications. When doing data analysis, the first and second moment constraints usually are enough to remove the selection bias. This point is also elaborated in Chan et al. (2016), who used a nonparametric assumption on propensity score. Furthermore, the smooth conditions on propensity score are also difficult to check in applications. That makes the proof of the theorems more complicated and requires the addition of other conditions which are equally difficult to check. For these reasons we added the propensity score assumptions.

The method can be easily extended to more than two treatment arms. The EL method can be applied to continuous outcomes or binary outcomes. It can also be modified to estimate average treatment effect in ATT, where ATT is the mean difference of two potential outcomes over the subpopulation of individuals who received the active treatment (Heckman et al., 1997; Imai and Ratkovic, 2014).

In principle, one can extend the estimator of Xie and Liu (2005) to a double-robustness estimator (augmented inverse probability weighted method, AIPW) using the general method suggested by Robins et al. (1994) (see also Lunceford and Davidian (2004); Tsiatis (2006)). The AIPW estimator(augmented inverse probability weighted method) is doubly robust in that the estimator is consistent when either the propensity score model is correctly specified or the model for the outcome as a function of the covariates is correctly specified. However, Kang and Schafer (2007) conducted a set of simulation studies to study the performance of propensity score weighting methods. They found that mis-specification of a propensity score model can negatively affect the performance of various weighting methods. In particular, they showed that although the doubly robust estimator of Robins et al. (1994) provides a consistent estimate of the treatment effect if either the outcome model or the propensity score model is correct, the performance of the doubly robust estimator can deteriorate when both models are slightly misspecified. In this paper, we have not discussed the semiparametric efficiency of the empirical likelihood method, though our simulation demonstrated efficiency compared to the propensity score-based method. Some authors, including Hirano et al. (2003), Qin and Zhang (2007), Hainmueller (2011), Chan et al. (2012), Chan et al. (2016), have discussed the semiparametric efficiency of empirical likelihood-based estimators in different but related settings. Whether our Kaplan-Meier estimator reaches the semiparametric bound and at what rate are topics for future research.