Quantifying treatment effects using the personalized chance of longer survival

Abstract

The hazard ratio (HR) is widely used to measure or to summarize the magnitude of treatment effects, but it is justifiably difficult to interpret in a meaningful way to patients and perhaps for clinicians as well. In addition, it is most meaningful when the hazard functions are approximately proportional over time. We propose a new measure, termed personalized chance of longer survival. The measure, which quantifies the probability of living longer with one treatment over the another, accounts for individualized characteristics to directly address personalized treatment effects. Hence, the measure is patient-focused, can be used to evaluate subgroups easily. We believe it is intuitive to understand and clinically interpretable in the presence of non-proportionality. Furthermore, because it estimates the probability of living longer by some fixed amount of time, it encodes the probabilistic part of treatment effect estimation. We provide nonparametric estimation and inference procedures that can accommodate censored survival outcomes. We conduct extensive simulation studies, which characterize performance of the proposed method and data from a large randomized Phase III clinical trial (SWOG S0819) is analyzed using the proposed method

1. Introduction

There has been a paradigm shift in oncology treatment. One focus of anticancer drug development is targeted therapies, which act on specific molecular targets to block the growth and spread of cancer. Usually, a subgroup of patients who have an appropriate target would benefit from a particular targeted therapy. Hence, targeted therapy has become the foundation of precision medicine, where individual information is used to prevent, diagnose and treat disease. In addition, cancer immunotherapy (IO) has been developed, where therapeutic agents are used to target immune cells rather than cancer cells. The utility of IOs works through the complexity of the immune system and the factors needed to activate the immune system, provided there is sufficient time to allow the immune system to work on the cancer. Especially delayed benefits in treatment with IO could occur, given that IOs do not work directly on a patient’s tumor. The targeted nature of targeted therapies and the “indirect” effect of IO motivate the need for modernizing trial analysis to evaluate agents, where the types of personalized effects and non-proportionality are expected.

Time-to event endpoints, such as overall survival are typically considered the gold standard for evaluating treatment efficacy, and are widely used as the primary endpoint in oncology clinical trials. Treatment effects are usually quantified in terms of the hazard ratio (HR), the ratio of hazard rates between two treatment arms. It is most meaningful when the hazard functions are approximately proportional over time. If the assumption of proportional hazards is not met, the true HR could change over time and the calculated HR would not best reflect the treatment benefit (1). More importantly, the hazard ratio is sometimes hard to interpret, since the concept of hazard rate, representing the instantaneous rate of failure for the survivors, could be difficult to explain to patients and practitioners. Researchers have attempted to investigate treatment effects based on other interpretable quantities. For example, milestone survival is evaluated, which is defined as the Kaplan–Meier survival probability at a time point defined a priori. Such a milestone usually represents a clinically meaningful benchmark (2). However, the milestone does not necessarily represent long-term survival. This approach ignores all of the events occurring after the chosen time point (3). Other measures of treatment effect have been proposed, for example, the integrated survival probability difference over a prespecified time interval (4), and the ratio of restricted mean survival times (5). However, they have no straightforward application to the individual patient. As new cancer treatments such as targeted therapies become increasingly popular, determination and evaluation of the groups who derive the most benefit become more important and complicated. Benefiting greatly from The Cancer Genome Atlas project (6), and the rapid pace of molecular and cancer immunology research, the current focus of most therapeutic development in oncology is on targeted therapies for molecular biomarker-defined subgroups, immunotherapies in immune biomarker-defined subgroups, or combinations of these therapies, also in subgroups (7; 8). Given this focus, incorporation of robust identification and evaluation of the candidate subgroups who are likely to derive the most benefit within clinical trial design and analysis has become more important and complicated. Indeed, there have been methods developed for estimating personalized treatment effects from censored data. For example, some authors discussed construction of optimal individualized treatment rules using censored data (9; 10; 11). Tian et al (2014) developed a method to estimate interactions between a treatment and a large number of covariates for survival outcomes (12). Henderson et al (2018) focused on estimating heterogeneous treatment effects using a nonparametric accelerated failure time model (13). However, as discussed in (14), in today’s era with increasing costs of therapies and the associated toxicities, it is important to not only evaluate new treatments in terms of their statistical improvement over standard-of-care therapies but also to evaluate the therapies relative to benchmarks thought to be clinically meaningful. We note that none of these aforementioned methods focus on the evaluation of personalized treatment effects against a clinically meaningful amount and consistent with the current paradigm, it would be useful to augment the current approaches such that we can describe personalized clinically meaningful benefits.

Péron et al. (2016) (15) proposed a measure of treatment benefit, termed the net chance of a longer survival, to demonstrate the treatment effect in clinical trials. The net chance of a longer survival is defined as the probability that a random patient in the treatment group survives longer (by certain time) than a random patient in the control group minus the probability of the opposite situation (16; 15). For example, if the net chance were estimated to be 0.1, a random patient in the treatment group would have a 10% higher probability of a longer survival than a patient in the control group. However, their developments did not account for censored data. Simply ignoring the censored observations could lead to biased results. Furthermore, this measurement does not take into account the personal characteristics that might affect the net chance of a longer survival. It is likely that treatment may work on a subset of patients, even though the overall treatment benefit is not significant. Hence, a more comprehensive measurement is desirable to characterize the individualized treatment effect.

In this paper, we propose a more patient-oriented measure – personalized chance of a longer survival, an augmented measure based on the net chance of benefit developed in (15) which incorporates the variability of standard(control arm) treatment efficacy and censored observations to estimate overall and subgroup treatment effects. The measure answers a patient’s question: “For a patient like me, what is the net chance of surviving longer with new treatment than with standard care?” or, choosing a clinically meaningful benchmark of 6 months (for example) “What is the net chance of surviving at least 6 months longer with new treatment than with standard care?” Its population analog, chance of a longer survival, is also presented. We develop valid estimation and inference procedures for the proposed measures using kernel smoothers, which can handle the situation with censored data.

In Section 2, we define the general concepts of chance and personalized chance of a longer survival. In Section 3, we discuss the nonparametric estimation and inference methods for the proposed measures. In Section 4, we conduct extensive simulation studies to examine the empirical performance. A data example using data from the SWOG Cancer Research Network (formerly known as Southwest Oncology Group) S0819 trial is presented in Section 5, and Section 6 provides a discussion.

2. Personalized chance of a longer survival

Let T denote the overall survival, $X={\left(X_1,\dots ,X_p\right)}^T\in {ℝ}^p$ denote baseline characteristics, and A ∈ {0, 1} denote the binary treatment assigned with 1 indicating the investigational treatment and 0 indicating the standard of care/control arm treatment. Let T(a) denote the potential outcome (17; 18) under the treatment a, a = 0,1, and $\tilde{T}\left(0\right)$ is an independent copy of T(0). The chance of a longer survival by at least m months is defined as

$$\Delta (m)=P\{T(1)>T(0)+m\}-P\{\tilde{T}(0)>T(0)+m\}.$$

Here, the choice of possible values for m is constrained by τ, and the value of Δ(m) can range from −1 to 1. It is intuitive to set m = 0 in Δ(m), which directly quantifies the chance of a greater survival probability. However, simple statistical improvement of a new therapy may not be adequate. Physicians and the FDA are evaluating therapies for their durable and clinically meaningful effects as treatment efficacy can also coupled with a rise in the incidence of toxicities or high costs associated with treatment. The ability to evaluate scenarios with m > 0 facilitates evaluation of the additional benefit in survival probability against clinical benefit benchmarks. Given that different patients may have different preferences over the tradeoff between efficacy and toxicity, a comprehensive investigation of scenarios with different m values will be valuable for patients to make decisions. The choice of m will also depend on the specific disease setting. For example, in advanced lung cancer m = 0 may be adequate but in earlier stage breast cancer, it may be important to choose a larger m.

In defining Δ(m), the second term serves as an offset that reflects the variability of the survival time for patients receiving the control arm treatment. By using this offset term, the proposed quantity reflects the benefit in a longer survival solely due to the treatment. The usefulness of such adjustment depends on the tail behavior of the survival distribution. A longer and heavier tail distribution leads to a more important adjustment. Furthermore, it may not play a role if m is extremely large, as the probability will be close to 0. However, for small to moderate m, higher variability in the control arm will lead to a larger adjustment. If there is no variability of outcomes at all in the control arm, then we can completely attribute the chance of a better survival to the treatment arm. For example, while $P\{\tilde{T}(0)>T(0)\}\equiv 1/2$, $P\{\tilde{T}(0)>T(0)+m\}$ for moderate m > 0 is larger with higher variability in control arm.

Provided that we are using data from a randomized clinical trial, it is reasonable to assume that the potential outcomes are well defined. We can express the quantity in terms of the data generating model (19). It follows that expressing Δ(m) as a function of observable outcomes, the quantity is defined as:

$$\Delta (m)=P\left\{T_{1i}>T_{0j}+m\right\}-P\left\{T_{0k}>T_{0j}+m\right\},$$

where T_1i is the outcome of a patient i from treatment group, and T_0j and T_1j are the outcomes of patients j and k from control group respectively. This is the probability that a random patient in the treatment group survives longer than a random patient in the control group by at least m, minus the probability of a random patient in the control group survives longer than another random patient in the control group by at least m. Hence, the chance of a longer survival for a patient in the treatment group is adjusted by the probability that the patient could have a longer survival even in the control group.

Remark 1 Assume that the proportional hazards assumption holds. Furthermore, there are constant hazard rates in both arms, with hazard rate λ₁ in treatment arm A = 1 and λ₀ in treatment arm A = 0. This indicates there is a constant HR between the two arms, Γ = λ₁/λ₀. Then,

$$\Delta (m)=P\left\{T_{1i}>T_{0j}+m\right\}-P\left\{T_{0k}>T_{0j}+m\right\}=\left(\frac{1}{1+\text{Γ}}-\frac{1}{2}\right)e^{-{\text{λ}}_{1}m}=\frac{(1-\text{Γ})e^{-{\text{λ}}_{1}m}}{2(\text{Γ}+1)}.$$

For the special case of m = 0,

$$\Delta (0)=P\left\{T_{1i}>T_{0j}\right\}-P\left\{T_{0k}>T_{0j}\right\}=\frac{1-\text{Γ}}{2(\text{Γ}+1)}.$$

Remark 1 provides a way to determine the clinically meaningful difference for Δ(m). For example, assume that median survival in control group is 6 months. According to (14), relative improvements of at least 20% in median overall survival are necessary to deem improvement in outcome clinically meaningful. We set our goal to increase median survival to 8 months (hazard ratio Γ = 0.75, assuming exponential survival function). In this case, a meaningful m value is 2 months, and a clinically meaningful improvement is Δ(2) = 6%.

Δ(m) is referred to a random patient without accounting for his/her individual characteristics. We further propose the concept of the personalized chance of a longer survival, denoted as Δ(m;x). For a patient with covariate x,

$$\Delta (m;x)=P\{T(1)>T(0)+m|X=x\}-P\{\tilde{T}(0)>T(0)+m|X=x\}.$$

P{T(1) > T(0) + m|X = x} is the probability for a patient with characteristics x to be favorable to the treatment by m month, and and $P\left\{\tilde{T}\left(0\right)>T\left(0\right)+m|X=x\right\}$ describes the probability for a patient with covariate x to live longer by m months compared to another patient with covariate x when they are both receiving the control therapy. Denote the outcomes of a patient in treatment or control group as T₁ and T₀ respectively. With data from a randomized clinical trial,

$$\Delta (m;x)=P\left\{T_1>T_0+m|X=x\right\}-P\left\{{\tilde{T}}_0>T_0+m|X=x\right\},$$

where ${\tilde{T}}_0$ is an independent copy of T₀.

Remark 2 Assume that the Cox proportional hazards model holds for the data. In particular, for patient with covariate x in treatment group, the hazard function is

$$\text{λ}(t|x,a)={\text{λ}}_0(t)\text{exp}(\gamma a)\text{exp}\left(x^T\beta \right).$$

The cumulative baseline hazard function is denoted by Λ₀(t). Then,

$$\begin{gathered} P\left(T_1>T_0|X=x\right) \\ =\int \text{exp}{\left(-{\Lambda }_0(t)\right)}^{\text{exp}(\gamma )\text{exp}\left(x^T\beta \right)}{f}_{T_0}(t)dt \\ =\int \text{exp}{\left(-{\Lambda }_0(t)\right)}^{\text{exp}(\gamma )\text{exp}\left(x^T\beta \right)}\text{exp}{\left(-{\Lambda }_0(t)\right)}^{\text{exp}\left(x^T\beta \right)}{\text{λ}}_0(t)\text{exp}\left(x^T\beta \right)dt \\ =\frac{\text{exp}\left(x^T\beta \right)}{\left\{\text{exp}(\gamma )+1\right\}\text{exp}\left(x^T\beta \right)}\int {\text{λ}}_0(t)\left\{\text{exp}(\gamma )+1\right\}\text{exp}\left(x^T\beta \right)\text{exp}{\left\{-{\Lambda }_0(t)\right\}}^{\left\{\text{exp}(\gamma )+1\right\}\text{exp}\left(x^T\beta \right)}dt \\ =\frac{1}{\text{exp}(\gamma )+1}. \end{gathered}$$

Hence, Δ(0;x) is also a constant, where

$$\Delta (0;x)=\frac{1-\text{exp}(\gamma )}{2\left\{\text{exp}(\gamma )+1\right\}}.$$

Remarks 1 and 2 establish connection between the proposed concepts and the popular Cox proportional hazards model. Particularly, under proportional hazards assumptions we can estimate the personalized chance of a better survival using the Cox model, which is appealing in practice. Our development in later sections focuses on non-parametric estimation that does not require the assumption of proportional hazards for estimation and inference.

3. Estimation and inferences

3.1. Setup and pairwise comparison

Let τ be end of study. Let C denote the censoring time, which could go beyond τ, and assume that C and T are independent given (A,X). Our data consists of n independent identically distributed subjects, {Y_i = T_i ∧ C_i,δ_i = I(T_i ≤ C_i),X_i,A_i}, i = 1,…, n, where δ = I(T ≤ C) denotes the censoring indicator. We assume that C and T are independent given (X,A).

Assume the observed outcome from the treatment group is denoted by Y₁ and denoted by Y₀ from the control group. For any pair of individuals, we can compare their outcomes and define the pairwise indicator function for l = 0, 1 as

$$L\left(Y_{li},Y_{0j}\right)=\{\begin{array}{ll}1\hfill & \text{if }{Y}_{li}-Y_{0j}>m,\hfill \\ 0\hfill & \text{Otherwise.}\hfill \end{array}$$

The pairwise comparison is evident if the outcomes are binary or continuous (16). However, in the presence of censoring with a time-to-event outcome, the pairwise comparison could be uninformative, if either or both outcomes are censored. In the following section, we will propose nonparametric estimation and inference methods for both Δ(m) and Δ(m;x) based on the pairwise comparisons, where inverse probability of censoring weighting techniques are used to adjust for censored observations, and kernel smoothing is considered to derive personalized effects.

3.2. Estimating chance of a longer survival

If all patients are fully observed, i.e., Y_i = T_i, Δ(m) can be estimated by $\widehat{\Delta }(m)$, where

$$\widehat{\Delta }(m)=\frac{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}L\left(Y_{1i},Y_{0j}\right)}{n_0{n}_1}-\frac{{\sum }_{i=1}^{n_0}{\sum }_{j=1}^{n_0}L\left(Y_{0i},Y_{0j}\right)}{n_0^2},$$ (1)

with n₀ and n₁ as the number of subjects in treatment arms A = 0 and A = 1 respectively. To simplify notations, even though the pairs in the second term are different from the ones in the first, we keep the same indices notation. In estimating Δ(m), the second term serves as an offset that reflects the estimated variability of the survival time for patients in the control arm. Note that this quantity is a generalization of Mann–Whitney U statistic, which tries to estimate P(X > Y), where X,Y are two random variables generated from two distributions (16).

With the presence of censoring, (1) may lead to biased results if we simply drop all uninformative pairs. We weight each observed pair by the inverse probability of censoring weights. Hence, each observed pair would represent multiple pairs that are similar to those who might be censored. Let δ₁ and δ₀ denote the censoring indicator in group 1 and group 0 respectively, and S_C(t|x,a) = P(C > t|A = a,X = x) be the conditional treatment specific survival function for the censoring time given covariates x. Then,

$$\begin{gathered} E\left\{\frac{I\left({\delta }_1=1,{\delta }_0=1\right)L\left(Y_1,Y_0\right)}{S_C\left(Y_1|X,A\right)S_C\left(Y_0|X,A\right)}|X,A\right\} \\ =E\left\{\frac{I\left(C_1>T_1\right)I\left(C_0>T_0\right)\left(T_1-T_0>m\right)}{S_C\left(Y_1|X,A\right)S_C\left(Y_0|X,A\right)}|X,A\right\} \\ =P\left(T_1-T_0>m|X,A\right), \end{gathered}$$

where we have used the conditional independence of T and C given X,A. Consequently, Δ(m) can be estimated by

$$\widehat{\Delta }(m)=\frac{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}\frac{I\left({\delta }_{1i}=1,{\delta }_{0j}=1\right)L\left(Y_{1i},Y_{0j}\right)}{{\widehat{S}}_C\left(Y_{1i}|X_{1i},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}\frac{I\left({\delta }_{1i}=1,{\delta }_{0j}=1\right)}{{\widehat{S}}_C\left(Y_{1i}|X_{1i},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}-\frac{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}\frac{I\left({\delta }_{0l}=1,{\delta }_{0j}=1\right)L\left(Y_{0l},Y_{0j}\right)}{{\widehat{S}}_C\left(Y_{0l}|X_{0l},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}\frac{I\left({\delta }_{0l}=1,{\delta }_{0j}=1\right)}{{\widehat{S}}_C\left(Y_{0l}|X_{0l},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}.$$ (2)

3.3. Estimating personalized chance of a longer survival

The information on the patient with covariate x is usually limited, especially if it is not discrete, in the observed dataset. To estimate the personalized chance of a longer survival, we will need to borrow information from other subjects. Given that we can compare different pairs, we can incorporate each pair’s information with certain weight that represents the similarity between this pair and a patient of interest. For this purpose, we will use a kernel smoother to estimate Δ(m;x).

Let K be a kernel function and b_n be a bandwidth sequence. Define $K_{b_n}\left(x_0-X\right)=K\left\{\left(x_0-X\right)/b_n\right\}/b_n^p$, where p is the number of covariates. Without any censoring, we propose to use the kernel estimator as

$$\widehat{\Delta }(m;x)=\frac{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{1i},Y_{0j}\right)}{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)}-\frac{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}{K}_{b_n}\left(x-X_{0l}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{0l},Y_{0j}\right)}{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}{K}_{b_n}\left(x-X_{0l}\right)K_{b_n}\left(x-X_{0j}\right)}.$$ (3)

Here, we assume that the bandwidth parameters are the same for simplicity. The estimator incorporates all pairs information between two arms in the first term, accompanied with the weights $K_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)$, which represent how similar the X_1i and X_0j are relative to patient x. This subsequently represents the similarity between the pair (Y_1i,Y_0j) and (T₁,T₀) for patient x to some extent. Similarly, the second term incorporates all pairs information within the control group to estimate $P\left\{{\tilde{T}}_0>T_0+m|X=x\right\}$ in Δ(m;x). The following proposition characterizes how the bias and variance of $\widehat{\Delta }(m;x)$ depend on the bandwidth b_n.

Proposition 3.1 Assume that K(u) is symmetric about 0 and satisfies ∫ K(u)du = 1. Then,

$$bias\{\widehat{\Delta }(m;x)\}=\frac{b_n^2}{2}{\sigma }_K^{4}M(x)+O\left(b_n^4\right),$$

and

$$Var\{\widehat{\Delta }(m;x)\}=\frac{4R{(K)}^{2}v(x)}{n^2{b}_n^{2p}},$$

where $v(x)=var{\{\Delta (m;X)\}|}_{X=x}$, ${\sigma }_K^2=\int u^{2}K(u)du$, $R(K)=\int K{(u)}^{2}du$, and M(x) is a function of x related to the first and second derivatives of Δ(m;x) and the density function. Details of M(x) can be found in the proof available in the Appendix.

The average mean squared error of $\widehat{\Delta }(m;x)$ is of the order $O\left(b_n^4\right)+O\left(1/n^2{b}_n^{2p}\right)$. A requirement for the average mean squared error to decrease to zero is that b_n → 0 and $n{b}_n^p\to \infty$. The optimal bandwidth b_n has the order of n^−1/(p+2).

When data are subject to censoring, we propose the kernel estimator using inverse probability of censoring weighting techniques as

$$\widehat{\Delta }(m;x)=\frac{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}\frac{K_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)I\left({\delta }_{1i}=1,{\delta }_{0j}=1\right)I\left\{L\left(Y_{1i},Y_{0j}\right)=1\right\}}{{\widehat{S}}_C\left(Y_{1i}|X_{1i},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}{{\sum }_{i=1}^{n_1}{\sum }_{j=1}^{n_0}\frac{K_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)I\left({\delta }_{1i}=1,{\delta }_{0j}=1\right)}{({\widehat{S}}_C\left(Y_{1i}|X_{1i},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}-\frac{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}\frac{K_{b_n}\left(x-X_{0l}\right)K_{b_n}\left(x-X_{0j}\right)I\left({\delta }_{0l}=1,{\delta }_{0j}=1\right)I\left\{L\left(Y_{0l},Y_{0j}\right)=1\right\}}{{\widehat{S}}_C\left(Y_{0l}|X_{0l},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}{{\sum }_{l=1}^{n_0}{\sum }_{j=1}^{n_0}\frac{K_{b_n}\left(x-X_{0l}\right)K_{b_n}\left(x-X_{0j}\right)I\left({\delta }_{0l}=1,{\delta }_{0j}=1\right)}{({\widehat{S}}_C\left(Y_{0l}|X_{1i},1\right){\widehat{S}}_C\left(Y_{0j}|X_{0j},0\right)}}.$$

In general, the convergence rate of $\widehat{\Delta }(m;x)$ will not be affected by the inverse probability of censoring weighting estimator; if we use working models for estimating S_C(t|x,a) it yields a root-n convergence rate. For example, we can use the Cox proportional hazards model for S_C(t|x,a) (20). Let λ_Ci(t) denote the hazard function of censoring times for subject i, and let Z denote regressors constructed from X and A used for modeling C. Under the Cox model, ${\text{λ}}_{Ci}(t)={\text{λ}}_{C0}(t)\text{exp}\left({\beta }_C^T{Z}_{Ci}\right)$, where λ_C0(t) is the baseline hazard function for censoring. The estimator for β_C, say ${\widehat{\beta }}_C$, can be obtained by maximizing the partial likelihood. The Breslow estimator can be used for the cumulative baseline hazard function Λ_C0(t), denoted by ${\widehat{\Lambda }}_{C0}(t)$. An estimator of Λ_C(t|A_i,X_i), the cumulative hazard function of censoring time for subject i, is thus $\text{exp}\left({\widehat{\beta }}_C^T{Z}_{Ci}\right){\widehat{\Lambda }}_{C0}(t)$. An estimator for S_C(t|A_i,X_i) is ${\widehat{S}}_C\left(t|A_i,X_i\right)=\text{exp}{\left\{-{\widehat{\Lambda }}_{C0}(t)\right\}}^{\text{exp}\left({\widehat{\beta }}_C^T{Z}_{Ci}\right)}$.

3.4. Bandwidth selection

As for any nonparametric estimation problem, it is crucial to select the appropriate bandwidth b_n. Since the theoretical optimal bandwidth is difficult to derive, we propose a practical empirical method for choosing the best bandwidth. If the biomarker is discrete, the bandwidth can be simply chosen as a nonzero value. Basically, in that case, estimation uses the information from all subjects in the dataset who have the biomarker value as x for the estimating $\widehat{\Delta }(m;x)$. If the biomarker is continuous, we use the cross-validation method with $\widehat{\Delta }(m)$ as criterion to choose b_n for $\widehat{\Delta }(m;x)$ The estimated optimal bandwidth minimizes an average squared distance between the observed $\widehat{\Delta }(m)$, and the predicted counterparts from aggregated $\widehat{\Delta }(m;x)$ in the validation samples under K-fold cross-validation setting. We randomly split the data into K disjoint subsets of approximately equal sizes, denoted by {I_l,l = 1,…,K}. For X_i in each l, we estimate $\widehat{\Delta }\left(m;X_i\right)$ using data not in I_l with a given b_n. We denote this estimator as ${\widehat{\Delta }}_{(-l)}\left(m;X_i\right)$. We then calculate ${\widehat{\Delta }}_{(-l)}\left(m;{\mathcal{I}}_l\right)$ as

$${\widehat{\Delta }}_{(-l)}\left(m;{\mathcal{I}}_l\right)=\frac{1}{n_l}\sum _{i\in {\mathcal{I}}_l}{\widehat{\Delta }}_{(-l)}\left(m;X_i\right),$$

where n_l denotes the number of subjects in ${\mathcal{I}}_l.\widehat{\Delta }\left(m;{\mathcal{I}}_l\right)$ can be estimated with data from I_l using (2). We calculate the square error ${\left\{\widehat{\Delta }\left(m;{\mathcal{I}}_l\right)-{\widehat{\Delta }}_{(-l)}\left(m;{\mathcal{I}}_l\right)\right\}}^2$ for the l^th subset. Last, we sum the square error over l = 1,…,K and then choose b_n by minimizing the sum over the K-folds.

3.5. Bootstrap methods for constructing confidence intervals

It is difficult to derive this limiting distribution in our case. However, the limiting distribution of $\widehat{\Delta }(m;x)$ can be approximated using the bootstrap. It was shown in (21) that limiting distribution of kernel regression estimates may be closely approximated by a Gaussian process under mild condition. Though the kernel type estimator does not achieve root-n consistency, the bootstrap can be shown to consistently approximate the limiting distribution of the personalized net benefit function, using arguments such as those given in (22).

Let (Y*,δ*,X*,A*) denote a resampling of (Y,δ,X,A) drawn randomly. The bootstrap estimate and ${\widehat{\Delta }}^{*}(m;x)$ are calculated based on the resample, in which (Y,δ,X,A) are replaced by (Y*,δ*,X*,A*). We repeat this process M times, and obtain M values of ${\widehat{\Delta }}^{*}(m;x)$, denoted as ${\widehat{\Delta }}_1^{*}(m;x),\dots ,{\widehat{\Delta }}_M^{*}(m;x)$. We then obtain the bootstrap percentile method (1 – α) confidence interval as $\left[{\widehat{\Delta }}_{\lfloor \alpha M/2\rfloor }^{*}(m;x),{\widehat{\Delta }}_{\lfloor M-\alpha M/2\rfloor }^{*}(m;x)\right]$, where $\lfloor t\rfloor$ is the largest integer that is smaller than t. The confidence interval for $\widehat{\Delta }(m)$ can be obtained similarly. Hall et al (2001) showed that there is no general first-order theoretical improvement in the accuracy of bootstrap approximations when using resampled bandwidth compared to the sample version. However, the computational burden is much higher (23). Hence, we recommend not to vary the bandwidth across resamples in the implementation.

4. Simulation Studies

We conducted simulation studies to illustrate our proposed method under differing treatment survival models. We generated treatment variable A∈ {0, 1} with a probability of 0.5. The survival times for patients in the control group were generated from a Weibull distribution, which is sufficiently flexible to approximate various survival curves observed in cancers. The Weibull model is given by S(t) = exp(−λt^k), where λ is called the scale parameter and k is called the shape parameter. For the control arm, we set λ₀ = 0.295 and k = 2. In Scenario 1, survival times in the treatment group followed a Weibull distribution with k = 2 and λ₁ = 0.221. The hazard ratio was constant and equal to 0.75. The proportional hazard assumption does not hold in Scenarios 2 & 3. In Scenario 2, survival times in the treatment group followed a Weibull distribution with k = 2 and λ₁ changed such that the HR was 0.4 between 0 and 4 months, 0.55 between 4 and 8 months, 0.7 between 8 and 12 months, 0.85 between 12 and 16 months, and to 1 after 16 months. This corresponds to the case where there is better survival in the treatment arm in the beginning, but the treatment benefit disappears at later times after registration/randomization. In Scenario 3, survival times in the treatment group followed a Weibull distribution with k = 2 and λ₁ such that the hazard ratio was 1 between 0 and 4 months. It decreased to 0.875 between 4 and 8 months, to 0.7 between 8 and 12 months, to 0.5 between 12 and 16 months and to 0.3 thereafter. This represents the case where there exists a delayed effect, and treatment works better in a later stage. The delayed separation has been observed in recent clinical trials with IOs (24; 25). See Figure 1 for the presentation of data generated under Scenarios 1–3.

Figure 1.

Kaplan-Meier curves for, left panel: Scenario 1; middle panel: Scenario 2; right panel: Scenario 3

We then considered cases where there existed a subgroup, and the treatment effects could differ between subgroups. We generated a binary covariate X ∈ {0, 1} with equal probabilities. Patients are randomized with equal probabilities to treatment (A = 1) and control (A = 0). In Scenario 4, for patients with X = 0, both survival times in treatment and control were generated from Weibull with λ₀ = λ₁ = 0.295 and k = 2. Data for patients with X = 1 were generated according to Scenario 2. In Scenario 5, survival times for patients with X = 0 in the treatment group followed a Weibull distribution with k = 2 and λ₁ = 0.266. The hazard ratio was equal to 0.9. Hence, there is a moderate treatment benefit. Survival times for patients with X = 1 were simulated following Scenario 3. In Scenario 6, the survival times for patients with X = 0 were generated using a constant hazard ratio equal to 2.5 (detrimental effect of the experimental treatment), and the survival times for patients with X = 1 were generated using a constant hazard ratio equal to 0.4 (benefit of the experimental treatment). See Figure 2 for the presentation of data generated under Scenarios 4–6.

Figure 2.

Kaplan-Meier curves for, left panel: Scenario 4; middle panel: Scenario 5; right panel: Scenario 6

Finally in Scenario 7, we investigated the case with a continuous biomarker, where X was generated from a standard normal distribution. The survival time T is the minimum of τ = 50 and $\tilde{T}$, which is generated with hazard rate function

$${\text{λ}}_{\tilde{T}}(t|X,A)=\text{exp}\{-1+(-0.25+X)A\}.$$

In all scenarios, we considered two censoring mechanisms. The censoring time C was generated from a uniform distribution on [0, 75] under censoring mechanism 1. With censoring mechanism 2, C depends on the covariate X, which is the minimum of 120 and the time generated with a hazard rate function e^2X/20.

A data set was generated including 2 treatment groups for each scenario, with the total sample size being 500 or 1000. We compared the proposed method with the method ignoring censoring in estimating the chance of a longer survival with m = 0. The biases and the associated 95% confidence intervals out of 500 replications are shown in Tables 1 and 2, along with the true adjusted probability of patients living longer on treatment. The confidence intervals were constructed based on 500 bootstrap samples. It can be seen that ignoring censoring could lead to large biases and severe under coverage in certain situations. The proposed method had much smaller biases and achieved nominal coverage most of the time, regardless of the censoring mechanism. And the performance improved as sample sizes increased.

Table 1.

Simulation results for chance of a longer survival for n = 500

Case	Truth	Censoring Type	Bias		95% CI
			Proposed	Ignore Censoring	Proposed	Ignore Censoring
1	0.071	1	0.004	0.019	0.932	0.896
1	0.071	2	0.006	0.017	0.93	0.916
2	0.055	1	0.005	0.008	0.932	0.938
2	0.055	2	0.006	0.007	0.93	0.95
3	0.161	1	0.004	0.062	0.932	0.414
3	0.161	2	0.005	0.055	0.932	0.522
4	0.041	1	0	0.009	0.95	0.944
4	0.041	2	0	0.012	0.946	0.932
5	0.081	1	0	0.032	0.952	0.796
5	0.081	2	0.001	0.046	0.946	0.644
6	0	1	0.004	0.009	0.924	0.94
6	0	2	0.003	0.043	0.932	0.654
7	0.022	1	0.004	−0.012	0.938	0.942
7	0.022	2	0.012	0.032	0.942	0.735

Table 2.

Simulation results for chance of a longer survival for n = 1000

Case	Truth	Censoring Type	Bias		95% CI
			Proposed	Ignore Censoring	Proposed	Ignore Censoring
1	0.071	1	0.001	0.02	0.952	0.842
1	0.071	2	0.002	0.019	0.938	0.814
2	0.055	1	0.002	0.008	0.954	0.95
2	0.055	2	0.003	0.009	0.938	0.912
3	0.161	1	0.001	0.063	0.966	0.096
3	0.161	2	0.002	0.058	0.944	0.166
4	0.041	1	0.001	0.009	0.944	0.932
4	0.041	2	0.001	0.011	0.948	0.918
5	0.081	1	0.001	0.032	0.94	0.624
5	0.081	2	0	0.045	0.954	0.362
6	0	1	0.001	0.011	0.934	0.896
6	0	2	0.001	0.045	0.96	0.342
7	0.022	1	0	−0.013	0.964	0.884
7	0.022	2	0.008	0.03	0.914	0.558

We estimated the personalized chance of a longer survival using the proposed method for patients with X = x₁ and X = x₂, where x₁ = 1 and x₂ = 0 in Scenarios 1–6, and x₁ and x₂ were set to the 75 and 25 percentiles of the biomarker X in Scenario 7. The results are presented in Tables 3 and 4. The true personalized chance of patients living longer were the same for x₁ and x₂ in Scenarios 1–3, since there were no subgroups present in the data. They differed in Scenarios 4–7 given that there was an interaction between the biomarker and the treatment in the generative model for the survival time. The proposed method gave good performance, although there was slight under coverage for some scenarios with n = 500 perhaps due to limited observations with biomarker values in the neighborhood of x₁ (x₂). Such phenomena disappeared when the sample size was increased to 1000.

Table 3.

Simulation results for personalized chance of a longer survival for x₁

Case	Truth	Censoring Type	Bias		95% CI
			n = 500	n = 1000	n = 500	n = 1000
1	0.071	1	0.007	0.001	0.936	0.966
1	0.071	2	0.012	0.002	0.934	0.944
2	0.055	1	0.008	0.001	0.926	0.962
2	0.055	2	0.013	0.003	0.934	0.958
3	0.161	1	0.007	0	0.916	0.962
3	0.161	2	0.01	0.001	0.934	0.928
4	0.055	1	0.002	0.001	0.948	0.942
4	0.055	2	0.006	0.004	0.934	0.944
5	0.161	1	0.003	0	0.936	0.956
5	0.161	2	0.003	0.003	0.934	0.948
6	0.214	1	0.003	0.001	0.938	0.93
6	0.214	2	0.004	0.001	0.914	0.95
7	−0.089	1	0.023	0.015	0.914	0.932
7	−0.089	2	0.021	0.014	0.935	0.91

Table 4.

Simulation results on personalized chance of a longer survival for x₂

Case	Truth	Censoring Type	Bias		95% CI
			n = 500	n = 1000	n = 500	n = 1000
1	0.071	1	0.007	0.004	0.946	0.942
1	0.071	2	0.006	0.005	0.94	0.92
2	0.055	1	0.008	0.005	0.952	0.93
2	0.055	2	0.006	0.005	0.946	0.914
3	0.161	1	0.006	0.004	0.938	0.94
3	0.161	2	0.006	0.005	0.946	0.928
4	0.026	1	0.003	0.001	0.94	0.934
4	0.026	2	0.001	0.001	0.944	0.942
5	0	1	0.003	0.001	0.948	0.924
5	0	2	0.001	0.001	0.944	0.942
6	−0.214	1	0.01	0.004	0.944	0.942
6	−0.214	2	0.008	0.003	0.938	0.946
7	0.151	1	0.005	−0.004	0.945	0.97
7	0.151	2	−0.004	−0.007	0.962	0.962

5. Data analysis: SWOG S0819 trial

SWOG S0819 was an open-label, phase 3 study evaluating the addition of cetuximab to standard-of-care chemotherapy, with bevacizumab, as appropriate, as first-line treatment for advanced non-small cell lung cancer (NSCLC). Cetuximab, an EGFR-directed monoclonal antibody, blocks ligand-induced EGFR activation, stimulates receptor internalization, and is capable of inducing antibody-dependent cellular cytotoxicity. The design of this study has been previously described (26) and the results published (27). Prior to the launch of this trial, there was evidence that cetuximab may have broad activity within this population, also hypothesized that the activity may be greater within subsets of patients based on EGFR expression. The goal of the study was to evaluate if the addition of cetuximab resulted in a longer duration to progression (progression-free survival) among patients defined to be EGFR positive based on the fluorescence in situ hybridization (FISH) assay, while the study enrolled an unselected population to evaluate the activity in this broader population as a coprimary objective. The primary endpoint for the entire study population evaluation was overall survival (OS). The sample size was based on attaining a specific number of EGFR FISH positive patients. In all, 1313 patients were randomized with 657 patients in the control and 656 in the cetuximab-containing arm; 400 EGFR FISH positive patients were enrolled. The study was closed at an interim analysis, meeting the pre-specified rules for futility. PFS was not significantly different in the EGFR FISH positive group and OS did not differ in the entire study population (HR 0.93, 95% CI 0.83–1.04; p=0.22; median 10.9 months [95% CI 9.5–12.0] vs 9.2 months [8.7–10.3]). In addition the evaluation within EGFR FISH positive patients, a pre-specified subgroup analysis based on the results of the SQUIRE trial (28) evaluated OS among EGFR positive patients with squamous histology. This analysis found a benefit within this subgroup (HR 0.58, 95% CI 0.36–0.86; p=0.0071). There were no significant differences and overall survival did not differ in other subgroups.

We applied the proposed measure on data from overall trial. We considered four subgroups of interest based on the histology type and FISH status: x = (1, 1) indicating FISH positive with squamous cell histology (111 patients), x = (1, 0) indicating FISH positive with non-squamous cell histology (289 patients), x = (0,1) indicating FISH negative with squamous cell histology (210 patients) and x = (0, 0) indicating FISH negative with non-squamous cell histology (703 patients). We present the chance and personalized chance of a better survival in Figure 3. The chance of a longer survival in the overall population is around 0 across all values of m, suggesting that the probability of living longer on treatment is not any better than the control. However, patients with FISH positive status and squamous cell histology enjoy significant benefits from the treatment, where the probability of living longer on treatment by 1 year is close to 10%. The 95% confidence intervals, calculated via bootstrap method described earlier, are shown in Table 5. We observe that as m increase, the confidence intervals are narrow. This is because P{T(1) > T(0) + m} and $P\left\{\tilde{T}\left(0\right)>T\left(0\right)+m\right\}$ decrease over increasing m as well as the variances of the estimated probabilities $\mathrm{\Delta }(m)=P\left\{T\left(1\right)>T\left(0\right)+m\right\}-P\left\{\tilde{T}\left(0\right)>T\left(0\right)+m\right\}$. In addition, when the m is large enough, Δ(m) is effectively equal to 0. There is no observation reflecting m difference in months in the empirical dataset. Subsequently we obtain a boostrap estimated confidence interval of (0, 0). There is a practical limitation on empirical estimation of confidence intervals with large m and limited follow-up.

Figure 3.

Chance and personalized chance of a longer survival in S0819

Table 5.

95% confidence intervals for chance and personalized chance of a longer survival in S0819

m	Δ(m)	Δ(m; x = (1, 1))	Δ(m; x = (1, 0))	Δ(m; x = (0, 1))	Δ(m; x = (0, 0))
0	(−0.056,0.057)	(0,0.375)	(−0.082,0.155)	(−0.131,0.126)	(−0.085,0.059)
3	(−0.063,0.058)	(−0.065,0.368)	(−0.101,0.154)	(−0.16,0.115)	(−0.098,0.068)
6	(−0.067,0.06)	(−0.058,0.391)	(−0.092,0.156)	(−0.158,0.126)	(−0.105,0.069)
9	(−0.071,0.056)	(−0.083,0.346)	(−0.092,0.146)	(−0.13,0.137)	(−0.122,0.054)
12	(−0.076,0.05)	(−0.083,0.316)	(−0.085,0.134)	(−0.135,0.129)	(−0.133,0.033)
15	(−0.079,0.041)	(−0.062,0.279)	(−0.088,0.116)	(−0.116,0.138)	(−0.137,0.024)
18	(−0.081,0.034)	(−0.045,0.258)	(−0.077,0.098)	(−0.112,0.131)	(−0.131,0.016)
21	(−0.074,0.032)	(−0.037,0.204)	(−0.075,0.076)	(−0.07,0.135)	(−0.126,0.018)
24	(−0.068,0.032)	(−0.026,0.178)	(−0.074,0.045)	(−0.053,0.143)	(−0.116,0.02)
27	(−0.063,0.031)	(0,0.153)	(−0.07,0.016)	(−0.023,0.131)	(−0.103,0.029)
30	(−0.056,0.027)	(0,0.099)	(−0.055,0.01)	(−0.011,0.127)	(−0.088,0.033)
33	(−0.047,0.026)	(0,0.038)	(−0.039,−0.003)	(−0.005,0.104)	(−0.073,0.037)
36	(−0.039,0.03)	(0,0)	(−0.021,0)	(0,0.093)	(−0.065,0.037)

6. Discussion

In this paper, we proposed new measures characterizing the treatment benefit, which are intuitive to understand and can be easily communicated to clinicians and practitioners. The overall measurement, chance of a longer survival, describes the treatment effects on the whole patient population; the personalized measurement, personalized chance of a longer survival, takes into account the individualized characteristics. Compared to the usual summary measures such as hazard ratio, the proposed quantities can be interpreted even when the proportional hazard assumption is violated.

As an alternative, the restricted mean survival time has been used to characterize treatment effects in some settings where proportional hazards assumptions done hold. It can be estimated based on areas between two treatment arm survival curves up to a fixed time point. Hence, the interpretation varies with different end time points. Again, the new method estimates the probability of living longer, or living at least m months longer could be a more direct and natural way of describing a patient’s expected experience. Therefore, we think it may more directly encodes the probabilistic part of treatment effect estimation. It likely also makes more sense to the physician attempting to explain the relative efficacy of a new therapy

Along with the new measures, we also presented new estimation and inference tools that are practically useful and theoretically valid. Note that even if we are only interested in estimating the overall measurement, the personalized measurements are still valuable, where we can aggregate the personalized measures over all X to estimate the chance of a longer survival. The proposed work should prove to be a useful tool that will aid in analysis and interpretation of treatment benefits in clinical trials.

There are two important future directions we are pursuing. First, we would like to develop methods that can be used for power analysis and sample size calculation, which is important in clinical trial design. Although there are connections with hazard ratios in some situations, and the existing tools can be directly applied, a more general procedure is needed to plan ahead a sample size, which is sufficient to guarantee that we achieve the desired personalized chance of a longer survival on treatment. In this case, the goal is to determine a minimal sample size such that

$$P\{|\widehat{\Delta }(m;x)-\Delta (m;x)|\le d\}\ge 1-\alpha ,$$

where α ∈ (0,1) and d > 0. We will need to derive the asymptotic distribution of $\widehat{\Delta }(m;x)$. As shown in (29), the kernel estimators are asymptotically normal. In addition, the variance of $\widehat{\Delta }(m;x)$ is of the order nb_n. Assume that $var\{\widehat{\Delta }(m;x)\}=C/n{b}_n$, and b_n = n^−r. Then the minimum sample size is

$${\left(\frac{C{d}^2}{z_{1-\alpha /2}^2}\right)}^{1-r},$$

where Z_α is the 100% α upper percentile of the standard normal distribution. Even if the calculations did not drive the primary study design Phase III trial sample size, which may be based on proportional hazards testing, the approximate variance calculations from above will be useful. They can be used to assess the prospective statistical properties personalized chance of living longer estimates from such a Phase III study as co-primary or key secondary objectives of the trial design. Second, individualized treatment rules (30; 31), tailoring treatment decisions as a function from patient characteristics to a recommended treatment, have become increasingly popular in recent years. Usually, the treatment rules are constructed by maximizing the mean of a pre-specified clinical outcome if applied to recommend treatments in a population of interest. It would be interested to develop treatment rules that utilize the proposed measure as a criterion.

Appendix. Proof of Proposition 3.1.

Let $\widehat{\Delta }(m;x)={\widehat{g}}_1(x)-{\widehat{g}}_2(x)$, where ${\widehat{g}}_1(x)$ and ${\widehat{g}}_2(x)$ correspond to the terms in (3). We first consider the bias of ${\widehat{g}}_1(x)$ relative to g₁(x) = g₁(x, x), where g₁(x₁,x₀) = P(Y₁ − Y₀ > m|X₁ = x₁,X₀ = x₀).

$$\begin{gathered} E\left[\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{1i},Y_{0j}\right)\right] \\ =\int \int \int \int I\left(y_1-y_0>m\right)\frac{1}{n_0{b}_n^p}\frac{1}{n_1{b}_n^p}K\left(\frac{x-u_1}{b_n}\right)K\left(\frac{x-u_0}{b_n}\right)f\left(u_1,u_0,y_1,y_0\right)d{u}_{1}d{u}_{0}d{y}_{1}d{y}_0 \\ =\int \int \int \int I\left(y_1-y_0>m\right)\frac{1}{n_0{b}_n^{p-1}}\frac{1}{n_1{b}_n^{p-1}}K\left(s_1\right)K\left(s_0\right)f\left(x-b_n{s}_1,x-b_n{s}_0,y_1,y_0\right)d{s}_{1}d{s}_{0}d{y}_{1}d{y}_0, \end{gathered}$$ (4)

using the change of variable s₁ = (x − u₁)/b_n and s₀ = (x − u₀)/b_n. Note that f(x − b_ns₁,x − b_ns₀,y₁,y₀) = f(y₁,y₀|x − b_ns₁,x − b_ns₀)f(x − b_ns₁,x − b_ns₀). The above quantity is equal to

$$\begin{gathered} \iint K\left(s_1\right)K\left(s_0\right)f\left(x-b_n{s}_1,x-b_n{s}_0\right)\iint I\left(y_1-y_0>m\right)f\left(y_1,y_0|x-b_n{s}_1,x-b_n{s}_0\right)d{s}_{1}d{s}_{0}d{y}_{1}d{y}_0 \\ =\iint K\left(s_1\right)K\left(s_0\right)f\left(x-b_n{s}_1,x-b_n{s}_0\right)Q\left(x-b_n{s}_1,x-b_n{s}_0\right)d{s}_{1}d{s}_0. \end{gathered}$$

Using Taylor series expansions for f(x − b_ns₁,x − b_ns₀) and g₁(x − b_ns₁,x − b_ns₀), we further obtain (4) equals to

$$f(x,x)g_1(x)+b_n^2{\sigma }_K^4\{f_{x_1}^{(1)}(x,x)g_{1,x_1}^{(1)}(x,x)+f_{x_2}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)+f_{x_1}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)+f_{x_2}^{(1)}(x,x)g_{1,x_1}^{(1)}(x,x)+f(x,x)g_{1,x_1}^{(2)}(x,x)+f(x,x)g_{1,x_2}^{(2)}(x,x)+f(x,x)g_{1,x_1}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)+f_{x_1}^{(2)}(x,x)g_1(x)+f_{x_2}^{(2)}(x,x)g_1(x)+2f_{x_1}^{(1)}(x,x)f_{x_2}^{(1)}(x,x)g_1(x)+o\left(h^2\right)\},$$

where ${\sigma }_K^2=\int u^{2}K(u)du$. Similarly,

$$\begin{gathered} E\left[\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)\right] \\ =f(x,x)+\frac{b_n^2}{2}{\sigma }_K^4\left\{f_{x_1}^{(2)}(x,x)+f_{x_2}^{(2)}(x,x)+2f_{x_1}^{(1)}(x,x)f_{x_2}^{(1)}(x,x)\right\}. \end{gathered}$$

Therefore,

$$\begin{gathered} E\left\{{\widehat{g}}_1(x)\right\}\approx \frac{f(x,x)\left[g_1(x)+b_n^2{\sigma }_K^4\{(*)\}\right]}{f(x,x)\left[1+b_n^2{\sigma }_K^4\left\{f_{x_1}^{(2)}+f_{x_2}^{(2)}+2f_{x_1}^{(1)}{f}_{x_2}^{(1)}\right\}/2f\right]} \\ =g_1(x)+\frac{b_n^2}{2}{\sigma }_K^{4}M(x), \end{gathered}$$

Where

$$M(x)=\{f_{x_1}^{(1)}(x,x)g_{1,x_1}^{(1)}(x,x)+f_{x_2}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)+f_{x_1}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)+f_{x_2}^{(1)}(x,x)g_{1,x_1}^{(1)}(x,x)+f(x,x)g_{1,x_1}^{(2)}(x,x)+f(x,x)g_{1,x_2}^{(2)}(x,x)+f(x,x)g_{1,x_1}^{(1)}(x,x)g_{1,x_2}^{(1)}(x,x)\},$$

using the approximation that (1 + h²c)⁻¹ ≈ 1 − h²c for small h. Hence, the result on bias of ${\widehat{g}}_1(x)$ follows.

Now,

$$\begin{gathered} Var\left[\frac{1}{n_1{n}_0}\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{1i},Y_{0j}\right)\right] \\ =\frac{1}{n_1{n}_0}E{\left[K_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{1i},Y_{0j}\right)\right]}^2-O\left(n^{-1}\right) \\ \approx \frac{R{(K)}^{2}f(x,x)}{n_0{n}_1{b}_n^{2p}}[g_1{(x)}^2+v_1(x)\}], \end{gathered}$$

where R(K) = ∫ K(u)²du, and v₁(x) = var{L(Y₁,Y₀)|X₁ = X,X₀ = X)}|_X=x. Also,

$$Var\left[\frac{1}{n_1{n}_0}\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)\right]\approx \frac{R{(K)}^{2}f(x,x)}{n_0{n}_1{b}_n^{2p}}.$$

Finally, Also,

$$Cov[\frac{1}{n_1{n}_0}\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right),\frac{1}{n_1{n}_0}\sum _{i=1}^{n_1}\sum _{j=1}^{n_0}{K}_{b_n}\left(x-X_{1i}\right)K_{b_n}\left(x-X_{0j}\right)L\left(Y_{1i},Y_{0j}\right)]\approx \frac{R{(K)}^{2}f(x,x)g_1(x)}{n_0{n}_1{b}_n^{2p}}.$$

Hence, using the formula for the variance of the ratio of two random variables,

$$Var\left\{{\widehat{g}}_1(x)\right\}\approx \frac{R{(K)}^2{v}_1(x)}{n_0{n}_1{b}_n^{2p}}\approx \frac{4R{(K)}^2{v}_1(x)}{n^2{b}_n^{2p}}.$$

The desired results thus follow. □