Estimation and testing in targeted goup sequential covariate-adjusted randomized clinical trials

Abstract

This article is devoted to the construction and asymptotic study of adaptive group sequential covariate-adjusted randomized clinical trials analyzed through the prism of the semipara-metric methodology of targeted maximum likelihood estimation (TMLE). We show how to build, as the data accrue group-sequentially, a sampling design which targets a user-supplied optimal design. We also show how to carry out a sound TMLE statistical inference based on such an adaptive sampling scheme (therefore extending some results known in the i.i.d setting only so far), and how group-sequential testing applies on top of it. The procedure is robust (i.e., consistent even if the working model is misspecified). A simulation study confirms the theoretical results, and validates the conjecture that the procedure may also be efficient.

1. Introduction

This article is devoted to the construction and asymptotic study of adaptive group sequential covariate-adjusted randomized clinical trials (RCTs) analyzed through the prism of the targeted maximum likelihood estimation (TMLE) methodology. The considered observed data structure writes as O = (W, A, Y) where W is a vector of baseline covariates, A is the treatment assignment and Y the primary outcome of interest. We assume that A is binary and that Y is one-dimensional. Typical parameters of scientific interest are Ψ₊ = E{E(Y|A = 1, W) – E(Y|A = 0, W)} (additive scale, which we consider hereafter) or Ψ_× = log E{E(Y|A = 1, W)} – log E{E(Y|A = 0, W)} (multiplicative scale). Such parameters can be interpreted causally whenever one is willing to assume the existence of a full data structure X = (W, Y(0), Y(1)) containing the vector of baseline covariates and the two counterfactual outcomes under the two possible treatments, and such that Y = Y (A) and A conditionally independent of X given W. If so indeed, Ψ₊ = EY (1) – EY (0) and Ψ_× = log EY (1) – log EY (0). Let us now explain what we mean by adaptive group sequential covariate-adjusted RCT.

Adaptive group sequential covariate-adjusted RCT.

By adaptive covariate-adjusted design, we mean in the setting of this article a clinical trial design that allows the investigator to dynamically modify its course through data-driven adjustment of the randomization probability based on data accrued so far, without negatively impacting the statistical integrity of the trial. Moreover, the patient’s baseline covariates may be taken into account for the random treatment assignment. This definition is slightly adapted from (Golub,2006). In particular, we assume with a view to the definition of pre-specified sampling plans given in (Emerson, 2006) that, prior to collection of the data, the trial protocol specifies the parameter of scientific interest, the inferential method and confidence level to be used when constructing a confidence interval for the latter parameter.

Furthermore, we assume that the investigator specifies beforehand in the trial protocol a criterion of special interest which yields a notion of optimal randomization scheme that we therefore wish to target. For instance, the criterion could translate the necessity to minimize the number of patients assigned to their corresponding inferior treatment arm, subject to level and power constraints. Or the criterion could translate the necessity that a result be available as quickly as possible, subject to level and power constraints. The sole restriction on the criterion is that it must yield an optimal randomization scheme which can be approximated from the data accrued so far. The two examples above comply with this restriction.

We focus in this article on targeted maximum likelihood estimation, as introduced by van der Laan and Rubin (2006) in the independent identically distributed (i.i.d) setting. The extension to the setting of adaptive RCTs was first considered in (van der Laan, 2008), upon which this article relies. In addition, we choose to consider specifically the second criterion cited above. Consequently, the optimal randomization scheme is the so-called Neyman allocation, which minimizes the asymptotic variance of the TMLE of the parameter of interest. We emphasize that there is nothing special about targeting the Neyman allocation, the whole methodology applying equally well to a large class of optimal randomization schemes derived from a variety of valid criteria.

By adaptive group sequential design, we refer to the possibility to adjust the randomization scheme only by blocks of c patients, where c ≥ 1 is a pre-specified integer (the case that c = 1 corresponds to a fully sequential adaptive design). The expression also refers to the fact that group sequential testing methods can be equally well applied on top of adaptive designs, an extension that we do not consider here. Although all our results (and their proofs) still hold for any c ≥ 1, we consider the case c = 1 in the theoretical part of the article for simplicity’s sake, but the case that c > 1 is considered in the simulation study.

Short bibliography.

The literature on adaptive designs is vast and our review is not comprehensive. Quite misleadingly, the expression “adaptive design” has also been used in the literature for sequential testing and, in general, for designs that allow data-adaptive stopping times for the whole study (or for certain treatment arms) which achieve the wished type I and type II errors requirements when testing a null hypothesis against its alternative.

Of course, data-adaptive randomization schemes have a long history that goes back to the 1930s, and we refer to Section 1.2 in (Hu and Rosenberger, 2006), Section 17.4 in (Jennison and Turnbull, 2000) and (Rosenberger, 1996) for a comprehensive historical perspective.

Many articles are devoted to the study of “response adaptive designs”, an expression implicitly suggesting that those designs only depend on past responses of previous patients and not on the corresponding covariates. We refer to (Hu and Rosenberger, 2006; Chambaz and van der Laan, 2011) for a bibliography on that topic. On the contrary, covariate-adjusted response adaptive (CARA) randomizations tackle the so-called issue of heterogeneity (i.e., the use of covariates in adaptive designs), by dynamically calculating the allocation probabilities on the basis of previous responses and current and past values of certain covariates. In this view, this article studies a new type of CARA procedure. The interest in CARA procedures is more recent, and there is a steadily growing number of articles dedicated to their study, starting with (Rosenberger et al., 2001; Bandyopadhyay and Biswas, 2001), then (Atkinson and Biswas, 2005; Zhang et al., 2007; Zha; Shao et al., 2010) among others. The latter articles are typically concerned with the convergence (almost sure and in law) of the allocation probabilities vector and of the estimator of the parameter in a correctly specified parametric model ((Shao et al., 2010) is devoted to the testing issue).

By contrast, the consistency and asymptotic normality results that we obtain in this article are robust to model misspecification. Thus, they notably contribute significantly to solving the question raised by the Food & Drug Administration (2006):

When is it valid to modify randomization based on results, for example, in a combined phase 2/3 cancer trial?

Finally, this article mainly relies on (Chambaz and van der Laan, 2009, 2011; van der Laan, 2008), the latter technical report paving the way to robust and more efficient estimation based on adaptive RCTs in a variety of other setting (including the case that the outcome Y is a possibly censored time-to-event).

Organization.

We first set the statistical framework in Section 2. The rationale of our adaptive covariate-adjusted designs is presented in Section 3, and we complete its formal definition in Section 4 where we also detail how the TMLE methodology operates. The asymptotic study is carried out in Section 5 (estimation) and in Section 6 (group sequential testing). The simulation study is developed in Section 7 (estimation) and in Section 8 (group sequential testing). The proofs are relegated to the Appendix.

2. Statistical framework

We tackle the asymptotic study of adaptive group sequential designs in the case of RCTs with covariate, binary treatment and one-dimensional primary outcome of interest. Thus, the observed data structure writes as O = (W, A, Y), where $W\in \mathcal{W}$ consists of some baseline covariates, $A\in \mathcal{A}=\{0,1\}$ denotes the assigned binary treatment, and $Y\in \mathcal{W}$ is the primary outcome of interest. For example, Y can indicate whether the treatment has been successful or not $(\mathcal{Y}=\{0,1\})$; or Y can count the number of times an event of interest has occurred under the assigned treatment during a period of follow-up $(\mathcal{Y}=\mathbb{N})$ or Y can measure a quantity of interest after a given time has elapsed $(\mathcal{Y}=ℝ)$ Although we will focus on the last case in this article, the methodology applies equally well to each example cited above.

Let us denote by P₀ the true distribution of the observed data structure O in the population of interest. We see P₀ as a specific element of the non-parametric set $\mathcal{M}$ of all possible observed data distributions. Note that, in order to avoid some technicalities, we assume (or rather: impose) that all elements of $\mathcal{M}$ are dominated by a common measure. The parameter of scientific interest is the marginal effect of treatment α =1 relative to treatment α = 0 on the additive scale, or risk difference: ${\psi }_0=E_P{}_{{}_0}\left\{E_P{}_{{}_0}\left[Y|A=1,W\right]–E_P{}_{{}_0}\left[Y|A=0,W\right]\right\}$. Of course, other choices such as the log-relative risk (the counterpart of the risk difference on the multiplicative scale) could be considered, and dealt with along the same lines. The risk difference can be interpreted causally for instance in the counterfactual framework.

For all $P\in \mathcal{M}$, let us introduce the shorthand notation Q_W(P)(W) = P[W], g(P)(A|W) = P[A|W], Q_Y|A,W(P)(O) = P[Y|A, W]. We use the alternative notation P = P_Q,g with Q = Q(P) ≡ (Q_w (P), Q_Y|A,W (P)) and g = g(P). Equivalently, P_Q,g is the data generating distribution such that Q(P_Q,g) = Q and g(P_Q,g) = g. In particular, we denote Q₀ = Q(P₀) = (Q_w(P₀), Q_Y|A,W(P₀)). We also introduce the notation $\mathcal{Q}=\{Q(P):P\in \mathcal{M}\}$ for the non-parametric set of all possible values of Q, and $\mathcal{G}=\{g(P):P\in \mathcal{M}\}$ for the non-parametric set of all possible values of g. Setting $\overline{Q}(P)(A,W)=E_P[Y|A,W]$ and ${\overline{Q}}_0=\overline{Q}(P_0)$ (with a slight abuse, we also write sometimes $\overline{Q}(Q)$ instead of $\overline{Q}(P_{Q,g})$), we define in greater generality

$$\Psi (P)=E_P\{\overline{Q}(P)(1,W)-\overline{Q}(P)(0,W)\}$$ (1)

over the whole set $\mathcal{M}$, so that ψ₀ equivalently writes as ψ₀ = Ψ(P₀). This notation also emphasizes the fact that Ψ(P) only depends on P through $\overline{Q}(P)$ and Q_W(P), justifying the alternative notation Ψ(P_Q,g) = Ψ(Q). The following proposition summarizes the most fundamental properties enjoyed by Ψ.

Proposition 1 (efficient influence curve). The functional Ψ is pathwise differentiable at every $P\in \mathcal{M}$ relative to the maximal tangent space. The efficient influence curve of Ψ at $P_{Q,g}\in \mathcal{M}$ is characterized by

$$D^{★}(P_{Q,g})(O)=D_1^{★}(Q)(W)+D_2^{★}(P_{Q,g})(O),$$ (2)

where

$$\begin{array}{lll}{D}_1^{★}(Q)(W)\hfill & =\hfill & \overline{Q}(1,W)-\overline{Q}(0,W)-\Psi (Q),and\hfill \\ D_2^{★}(P_{Q,g})(O)\hfill & =\hfill & \frac{2A-1}{g(A|W)}(Y-\overline{Q}(A,W)).\hfill \end{array}$$

The variance Var_pD^★(P)(O) is the lower bound of the asymptotic variance of any regular estimator of Ψ(P) in the i.i.d setting. Furthermore, even if Q ≠ Q₀,

$$\begin{array}{ccc}{E}_{P_0}{D}^{★}(P_{Q,g})(O)=0& implies& \Psi (Q)=\Psi (Q_0)\end{array}$$ (3)

when g = g(P₀).

The implication (3) is the key to the robustness of the targeted maximum likelihood introduced and studied in this article. It is another justification of our interest in the pathwise differentiability of the functional Ψ and its efficient influence curve.

3. Data generating mechanism for adaptive design

The purpose of adaptive group sequential design as we consider it in this article is to adjust the randomization scheme as the data accrue. We first formally describe in Section 3.1 the data generating mechanism through the expression of the likelihood function and also in terms of causal graphs. Then we discuss the general issue of choosing an optimal design to target in Section 3.2, before specifically describing an optimal design of interest in Section 3.3 and showing how to target it in Section 3.4.

3.1. Data generating mechanism and related likelihood

In order to formally describe the data generating mechanism, we need to state a starting assumption: During the course of the clinical trial, it is possible to recruit independently the patients from a stationary population. In the counterfactual framework, this is equivalent to supposing that it is possible to sample as many independent copies of the full data structure as required. Let us denote by O_i = (W_i,A_i,Y_i) the ith observed data structure. We also find convenient to introduce O_n = (O₁,…, O_n) and for every i = 0,…, n, O_n(i) = (O₁,…, O_i) (with convention O(0)= ∅). We denote by $\mathcal{O}$ the set where the observed data structure O takes its values.

By adjusting the randomization scheme as the data accrue, we mean that the nth treatment assignment A_n is drawn from g_n(·|W_n), where g_n(·|W) is a conditional distribution (or treatment mechanism) given the covariate W which additionally depends on past observations O_n−1. Since the sequence of treatment mechanisms cannot reasonably grow in complexity as the sample size increases, we will only consider data-adaptive treatment mechanisms such that g_n(·|W) depends on O_n−1 only through a finite-dimensional summary measure Z_n = ϕ_n (O_n−1), where the measurable function ϕ_n maps ${\mathcal{O}}^{n-1}$ onto ${ℝ}^d$ for some fixed d ≥ 0 (d = 0 corresponds to the case that g_n(·|W) actually does not adapt). For instance, $Z_{n+1}={\varphi }_{n+1}(O_n)\equiv (n^{-1}{\displaystyle {\sum }_{i=1}^n{Y}_{i}1\{A_i=0\}},n^{-1}{\displaystyle {\sum }_{i=1}^n{Y}_{i}1\{A_i=1\})}$ characterizes a proper summary measure of the past, which keeps track of the mean outcome in each treatment arm. Another sequence of mappings ϕ_n will be at the core of the adaptive methodology that we study in depth in this article, see (9).

Formally, the data generating mechanism is specified by the following factorization of the likelihood of O_n:

$${\displaystyle \prod _{i=1}^n\{Q_W(P_0)(W_i)\times Q_{Y|A,W}(P_0)(O_i)\}}\times {\displaystyle \prod _{i=1}^n{g}_i(A_i|W_i)},$$

which suggests the introduction of g_n = (g₁,…,g_n), referred to as the design of the study, and the expression “O_n is drawn from (Q₀, g_n)”. Likewise, the likelihood of O_n under (Q, g_n) (where $Q=(Q_W,Q_{Y|A,W})\in \mathcal{Q}$ is a candidate value for Q₀) is

$${\displaystyle \prod _{i=1}^n\{Q_W(W_i)\times Q_{Y|A,W}(O_i)\}}\times {\displaystyle \prod _{i=1}^n{g}_i(A_i|W_i)},$$ (4)

where we emphasize that the second factor is known. Thus we will refer with a slight abuse of terminology to ${\displaystyle {\sum }_{i=1}^n\text{log\hspace{0.17em}}{Q}_W(W_i)+\text{\hspace{0.17em}log\hspace{0.17em}}{Q}_{Y|A,W}(O_i)}$ as the log-likelihood of O_n under (Q, g_n). Furthermore, given g_n, we introduce the notation $P_{Q_{0,g_i}}f\equiv E_{P_{Q_{0,g_i}}}[f(O_i)|O_n(i-1)]$ for any possibly vector-valued measurable f defined on $\mathcal{O}$.

Another equivalent characterization of the data generating mechanism involves the causal graph of Fig. 1. It is seen again firstly that W_n is drawn independently from the past O_n−1; secondly that A_n is a deterministic function of W_n, the summary measure Z_n (which depends on O_n−1), and a new independent source of randomness (in other words, it is drawn conditionally on (W_n, Z_n) and conditionally independently of the past O_n−1); thirdly that Y_n is a deterministic function of (A_n, W_n) and a new independent source of randomness (in other words, it is drawn conditionally on (A_n,W_n) and conditionally independently of the past O_n−1); then that the next summary measure Z_n+1 is obtained as a function of O_n−1 and O_n = (W_n,A_n,Y_n) (i.e., as a function of O_n; here, the causal graph grants the access to a new independent source of randomness, but it is useless in our setting), and so on.

Figure 1: A causal graph describing the data generating mechanism for adaptive group sequential design.

An from nodes Λ₁,..., Λ_r to node ϒ means that there exists a deterministic function f and an independent source of randomness U such that ϒ = f (Λ₁, …, Λ_r, U).

Finally, it is interesting in practice to adapt the design group sequentially. This can be simply formalized. For a given pre-specified integer c ≥ 1 (c = 1 corresponds to a fully sequential adaptive design), proceeding c-group sequentially simply amounts to imposing ϕ_(r−1)c+1(O_(r−1)c) = … = ϕ_rc (O_rc−1) for all r ≥ 1. Then the c treatment assignments A_(r−1)c+1,…,A_rc in the rth c-group are all drawn from the same conditional distribution g_(r−1)c(·|W). Yet, although all our results (and their proofs) still hold for any c ≥ 1, we prefer to consider in the rest of this section and in Sections 4 and 5 the case that c = 1 for simplicity’s sake. On the contrary, the simulation study carried out in Section 7 involves some c > 1.

3.2. On the user-supplied design to target

One of the most important features of the adaptive group sequential design methodology is that it targets a user-supplied specific design of special interest. This specific design is generally an optimal design with respect to a criterion which translates what the investigator cares for the most. Specifically, one could care the most for the well-being of the target population, wishing that a result be available as quickly as possible and aspiring therefore to the highest efficiency (i.e., the ability to reach a conclusion as quickly as possible subject to level and power constraints). Or one could care the most for the well-being of the subjects participating to the clinical trial (therefore trying to minimize the number of patients assigned to their corresponding inferior treatment arms, subject to level and power constraints). Obviously, these are only two important examples from a large class of potentially interesting criteria. The sole purpose of the criterion is to generate a random element in $\mathcal{G}$ of the form g_n = gZ_n, where Z_n = ϕ_n (O_n−1) is a finite-dimensional summary measure of O_n−1.

We decide to focus in this article on the first example, but it must be clear that the methodology applies to a variety of other criteria (see (van der Laan, 2008) for other examples).

3.3. From the Neyman allocation to the optimal design

So, the objective is now clearly identified: we wish to adapt the design as the data accrue in order to learn from the data then mimic a specific design which guarantees the highest efficiency, our (user-supplied) optimal design.

By Proposition 1, the asymptotic variance of any regular estimator of the risk difference Ψ(Q₀) has lower bound ${\text{Var}}_{P_{Q_0,g}}{D}^{★}(P_{Q_{0,g}})(O)$ if the estimator relies on data sampled independently from $P_{Q_{0,g}}$. Now,

$${\text{Var}}_{P_{Q_0,g}}{D}^{★}(P_{Q_{0,g}})(O)=E_{Q_0}{({\overline{Q}}_0(1,W)-{\overline{Q}}_0(0,W)-\Psi (Q_0))}^2+E_{Q_0}\left(\frac{{\sigma }^2(Q_0)(1,W)}{g(1|W)}+\frac{{\sigma }^2(Q_0)(0,W)}{g(0|W)}\right),$$

where σ²(Q₀)(A, W) denotes the conditional variance of Y given (A, W) under Q₀. We use the notation E_Q0 above (for the expectation with respect to the marginal distribution of W under P₀) in order to emphasize the fact that the treatment mechanism g only appears in the second term of the right-hand side sum. Furthermore, it holds P₀-almost surely that

$$\frac{{\sigma }^2(Q_0)(1,W)}{g(1|W)}+\frac{{\sigma }^2(Q_0)(0,W)}{g(0|W)}\ge {(\sigma (Q_0)(1,W)+\sigma (Q_0)(0,W))}^2,$$

with equality if and only if

$$g(1|W)=\frac{\sigma (Q_0)(1,W)}{\sigma (Q_0)(1,W)+\sigma (Q_0)(0,W)}$$ (5)

P₀-almost surely. Therefore, the following lower bound holds for all $g\in \mathcal{G}$:

$${\text{Var}}_{P_{Q_{0,g}}}{D}^{★}(P_{Q_{0,g}})(O)\ge E_{Q_0}{({\overline{Q}}_0(1,W)-{\overline{Q}}_0(0,W)\Psi (Q_0))}^2+E_{Q_0}{(\sigma (Q_0)(1,W)+\sigma (Q_0)(0,W))}^2,$$

with equality if and only if $g\in \mathcal{G}$ is characterized by (5). This optimal design is known in the literature as the Neyman allocation (see (Hu and Rosenberger, 2006) page 13). This result notably makes clear that the most efficient treatment mechanism assigns with higher probability a patient with covariate vector W to the treatment arm such that the variance of the outcome Y in this arm is the largest, regardless of the mean of the outcome (i.e., whether the arm is inferior or superior).

Due to logistical reasons, it might be preferable to consider only treatment mechanisms that assign treatment in response to a subvector V of the baseline covariate vector W. In addition, if W is complex, targeting the optimal Neyman allocation might be too ambitious. Therefore, we will consider the important case where V is a discrete covariate with finitely many values in the set $\mathcal{V}=\{1,\dots ,v\}$. The covariate V indicates subgroup membership for a collection of ν subgroups of interest. We decide to restrict the search of an optimal design to the set ${\mathcal{G}}_1\subset \mathcal{G}$ of those treatment mechanisms which only depend on W through V. The same calculations as above yield straightforwardly that, for all $g\in {\mathcal{G}}_1$,

$${\text{Var}}_{P_{Q_{0,g}}}{D}^{★}(P_{Q_{0,g}})(0)\ge E_{Q_0}{({\overline{Q}}_0(1,W)-{\overline{Q}}_0(0,W)-\Psi (Q_0))}^2+E_{Q_0}{(\overline{\sigma }(Q_0)(1,V)+\overline{\sigma }(Q_0)(0,V))}^2,$$

where ${\overline{\sigma }}^2(Q_0)(a,V)=E_{Q_0}[{\sigma }^2(Q_0)(a,W)|V]$ for $a\in \mathcal{A}$ with equality if and only if g coincides with g^★(Q₀), characterized by

$$g^{★}(Q_0)(1|V)=\frac{\overline{\sigma }(Q_0)(1,V)}{\overline{\sigma }(Q_0)(1,V)+\overline{\sigma }(Q_0)(0,V)}$$ (6)

P₀-almost surely. Hereafter, we refer to g^★(Q₀) as the optimal design.

3.4. Targeting the optimal design

Because g^★(Q₀) is characterized as the minimizer over $g\in {\mathcal{G}}_1$ of the variance under $P_{Q_{0,}g}$ of the efficient influence curve at $P_{Q_{0,}g}$ we propose to construct $g_{n+1}\in {\mathcal{G}}_1$ as the minimizer over $g\in {\mathcal{G}}_1$ of an estimator of the latter variance based on past observations O_n.

We proceed by recursion. We first set g₁ = g^b, the so-called balanced treatment mechanism such that $g^b(1|W)=\frac{1}{2}$ for all $W\in \mathcal{W}$, and assume that O_n has already been sampled from (Q₀, g_n) as described in Section 3.1, the sample size being large enough to guarantee ${\displaystyle {\sum }_{i=1}^{n}1\{V_i=v\}>0}$ for all $v\in \mathcal{V}$ (if n₀ is the smallest sample size such that the previous condition is met, then we set $g_1=\dots =g_{n_0}=g^b$).

The issue is now to construct g_n+1. Let us assume for the time being that we already know how to construct an estimator Q_n of Q₀ based on O_n (hence the estimators ${\overline{Q}}_n=\overline{Q}(Q_n)$ of ${\overline{Q}}_0$ and Ψ(Q_n) of Ψ(Q₀) = ψ₀).¹ Then, for all $g\in {\mathcal{G}}_1$,

$$\begin{array}{l}{S}_n(g)=\frac{1}{n}{\displaystyle \sum _{i=1}^n\{(D_1^{★}(Q_n){(W_i)}^2+2D_1^{★}(Q_n)(W_i)D_2^{★}(P_{Q_n,g})(O_i)\frac{g(A_i|V_i)}{g_i(A_i|V_i)}}\\ \begin{array}{cc}\begin{array}{cc}& \end{array}& +D_2^{★}(P_{Q_n,g}){(O_i)}^2\frac{g(A_i|V_i)}{g_i(A_i|V_i)})-{\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{D}_1^{★}(Q_n)(W_i)+D_2^{★}(P_{Q_n,g})(O_i)\frac{g(A_i|V_i)}{g_i(A_i|V_i)}}\right)}^2\}\end{array}\\ =\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{{(Y_i-{\overline{Q}}_n(A_i,W_i))}^2}{g(A_i|V_i)g_i(A_i|V_i)}}\\ \begin{array}{ccc}\begin{array}{cc}\begin{array}{cc}& \end{array}& \end{array}& & +\{\frac{1}{n}{\displaystyle \sum _{i=1}^n{D}_1^{★}(Q_n){(W_i)}^2+2D_1^{★}(Q_n)(W_i)D_1^{★}(Q_n)(W_i)D_2^{★}(P_{Q_n,g_i})(O_i)}\end{array}\\ \begin{array}{cc}\begin{array}{cc}\begin{array}{cc}\begin{array}{cccc}\begin{array}{cccc}\begin{array}{cc}\begin{array}{cc}& \end{array}& \end{array}& & & \end{array}& & & \end{array}& \end{array}& \end{array}& -{\left(\frac{1}{n}{\displaystyle \sum _{i=1}^n{D}_1^{★}(Q_n)(W_i)+D_2^{★★}(P_{Q_n,g_i})(O_i)}\right)}^2\}\end{array}\end{array}$$

estimates $\text{Var\hspace{0.17em}}{P}_{Q_{0,}g}{D}^{{}^{★}}\left(O;P_{Q_0,g}\right)$ (the weighting provides the adequate tilt of the empirical distribution; it is not necessary to weight the terms corresponding to $D_1^{★}$ because they do not depend on the treatment mechanism). Now, only the first term in the rightmost expression still depends on g. The same calculations as above straightforwardly yields that S_n (g) is minimized at $g_{n+1}^{★}\in {\mathcal{G}}_1$ characterized by

$$g_{n+1}(1|v)=\frac{s_{v,n}(1)}{s_{v,n}(1)+s_{v,n}(0)},$$ (7)

for all $v\in \mathcal{V}$, where for each $(v,a)\in \mathcal{V}\times \mathcal{A}$,

$$s_{v,n}^2(a)=\frac{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\frac{{(Y_i-{\overline{Q}}_n(A_i,W_i))}^2}{g_i(A_i|V_i)}}\mathbf{1}\{(V_i,A_i)=(v,a)\}}{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\mathbf{1}\{V_i=v\}}}.$$

Yet, instead of considering the above characterization, we find it more convenient to define

$$g_{n+1}^{★}(1|v)=\frac{{\sigma }_{v,n}(1)}{{\sigma }_{v,n}(1)+{\sigma }_{v,n}(0)},$$ (8)

for all $v\in \mathcal{V}$, where for each $(v,a)\in \mathcal{V}\times \mathcal{A}$,

$${\sigma }_{v,n}^2(a)=\frac{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\frac{{(Y_i-{\overline{Q}}_n(A_i,W_i))}^2}{g_i(A_i|V_i)}\mathbf{1}\{(V_i,A_i)=(v,a)\}}}{\frac{1}{n}{\displaystyle {\sum }_{i=1}^n\frac{\mathbf{1}\{(V_i,A_i)=(v,a)\}}{g_i(A_i|V_i)}}}.$$

Note that $s_{v,n}^2(a)$ and ${\sigma }_{v,n}^2(a)$ share the same numerator, and that the different denominators converge to the same limit. Substituting ${\sigma }_{v,n}^2(a)$ for $s_{v,n}^2(a)$ is convenient because one naturally interprets the former as an estimator of the conditional variance of Y given (A, V) = (a, v) based on O_n, a fact that we use in Section 4.2. Finally, we emphasize that $g_{n+1}^{★}=g{z}_{n+1}$ for the summary measure of the past O_n

$$Z_{n+1}={\varphi }_{n+1}({\mathbf{O}}_n)\equiv (({\sigma }_{v,n}^2(0),{\sigma }_{v,n}^2(1)):v\in \mathcal{V}).$$ (9)

The rigorous definition of the design ${\mathbf{g}}_n^{★}=(g_1^{★},\dots ,g_n^{★})$ follows by recursion, but it is still subject to knowledge about how to construct an estimator Q_n of Q₀ based on O_n. Because this last missing piece of the formal definition of the adaptive group sequential design data generating mechanism is also the core of the TMLE procedure, we address it in Section 4.

4. TMLE in adaptive covariate-adjusted RCTs

We assume hereafter that O_n has already been sampled from the $(Q_0,g_n^{★})$-adaptive sampling scheme. In this section, we construct an estimator Q_n (actually denoted by $Q_n^{*}$) of Q₀, therefore yielding the characterization of $g_{n+1}^{★}$, and completing the formal definition of the adaptive design ${\mathbf{g}}_n^{★}$ that we initiated in Section 3. In particular, the next observed data structure O_n+1 can be drawn from $(Q_0,g_{n+1}^{★})$, and it makes sense to undertake the asymptotic study of the properties of the TMLE methodology based on adaptive group sequential sampling.

As in the i.i.d framework, the TMLE procedure maps an initial substitution estimator $\Psi (Q_n^0)$ of ψ₀ into an update ${\psi }_n^{*}=\Psi (Q_n^{*})$ by fluctuating the initial estimate $Q_n^0$ of Q₀. The construction of $Q_n^0$ is presented and studied in Section 4.1. In Section 4.2, a straightforward application of the main result of Section 4.1 shows that the adaptive design converges. How to fluctuate $Q_n^0$ and stretch it optimally to $Q_n^{*}$ is presented and studied in Section 4.3.

4.1. Initial maximum likelihood based substitution estimator Working model.

In order to construct the initial estimate $Q_n^0$ of Q₀, we consider a working model $Q_n^w$. With a slight abuse of notation, the elements of ${\mathcal{Q}}_n^w$ are denoted by (Q_W(P_n),Q_Y|A,W(θ)) for some parameter θ ∈ Θ, where Q_W(P_n) is the empirical marginal distribution of W. Specifically, the working model ${\mathcal{Q}}_n^w$ is chosen in such a way that

$$Q_{Y|A,W}(\theta )(O)=\frac{1}{\sqrt{2\pi {\sigma }_V^2(A)}}\text{exp}\left\{-\frac{{(Y-m(A,W;{\beta }_V))}^2}{2{\sigma }_V^2(A)}\right\}.$$

This notably implies that for any $P_{\theta }\in \mathcal{M}$ such that Q_Y|A,W (P_θ) = Q_Y|A,W (θ), the conditional mean $\overline{Q}(P_{\theta })(A,W)$, which we also denote by $\overline{Q}(\theta )$, satisfies $\overline{Q}(\theta )(A,W)=m(A,W;{\beta }_V)$, the right-hand side expression being a linear combination of variables extracted from (A, W) and indexed by the regression vector β_V (of dimension b). Defining

$$\theta (v)={({\beta }_v,{\sigma }_v^2(0),{\sigma }_v^2(1))}^{\top }\in {\Theta }_v\subset {ℝ}^b\times {ℝ}_{+}^{*}\times {ℝ}_{+}^{*}$$ (10)

for each $v\in \mathcal{V}$, the complete parameter is given by θ = (θ(1)^⊤,…,θ(ν)^⊤)^⊤ ∈ Θ, where $\Theta ={\displaystyle {\prod }_{v=1}^{\nu }{\Theta }_v}$. We impose the following condition on the parameterization:

PARAM. The parameter set Θ is compact. Furthermore, the linear parameterization is identifiable: for all $v\in \mathcal{V}$, if $m(a,w;{\beta }_v)=m(a,w;{\beta }_v^{\prime })$ for all $a\in \mathcal{A}$ and $w\in \mathcal{W}$ (compatible with v), then necessarily ${\beta }_v={\beta }_v^{\prime }$.

Characterizing $Q_n^0$.

Let us set a reference fixed design $g^r\in {\mathcal{G}}_1$. We now characterize $Q_n^0$ by letting

$$Q_n^0=(Q_W(P_n),Q_{Y|A,W}({\theta }_n)),$$ (11)

Where

$${\theta }_n=\underset{\theta \in \Theta }{\text{arg\hspace{0.17em}max}}{\displaystyle \sum _{i=1}^n\text{log\hspace{0.17em}}{Q}_{Y|A,W}(\theta )(O_i)\frac{g^r(A_i|V_i)}{g_i^{★}(A_i|V_i)}}$$ (12)

is a weighted maximum likelihood estimator with respect to the working model. Thus, the vth component θ_n(v) of θ_n satisfies

$${\theta }_n(v)=\underset{\theta (v)\in {\Theta }_v}{\text{arg\hspace{0.17em}min}}{\displaystyle \sum _{i=1}^n\left(\text{log\hspace{0.17em}}{\sigma }_v^2(A_i)+\frac{{(Y_i-m(A_i,W_i;{\beta }_v))}^2}{{\sigma }_v^2(A_i)}\right)}\frac{g^r(A_i|V_i)}{g_i^{★}(A_i|V_i)}\mathbf{1}\{V_i=v\}$$

for every $v\in \mathcal{V}$. Note that this initial estimate $Q_n^0$ of Q₀ yields the initial maximum likelihood based substitution estimator $\Psi (Q_n^0)$ of ψ₀:

$$\Psi (Q_n^0)=\frac{1}{n}{\displaystyle \sum _{i=1}^n\overline{Q}({\theta }_n)(1,W_i)-\overline{Q}({\theta }_n)(0,W_i)}.$$

Studying $Q_n^0$ through θ_n.

For simplicity, let us introduce, for all θ ∈ Θ, the additional notations

$$\begin{array}{ccc}{\mathcal{l}}_{\theta ,0}=\text{log\hspace{0.17em}}{Q}_{Y|A,W}(\theta ),& {\dot{\mathcal{l}}}_{\theta ,0}=\frac{\partial }{\partial \theta }{\mathcal{l}}_{\theta ,0},& {\ddot{\mathcal{l}}}_{\theta ,0}=\frac{{\partial }^2}{\partial {\theta }^2}{\mathcal{l}}_{\theta ,0}.\end{array}$$

The first asymptotic property of θ_n that we derive concerns its consistency (see Theorem 5 in (van der Laan, 2008)).

Proposition 2 (consistency of θ_n). Assume that:

A1. There exists a unique interior point θ₀ ∈ Θ such that θ₀ = arg max_θ∈Θ P_Q0,g^rℓ_θ,0.

A2. The matrix – $P_{Q_{0,g^r}}{\ddot{\mathcal{l}}}_{{\theta }_0,0}$ is positive definite.

Provided that $\mathcal{O}$ is a bounded set, θ_n consistently estimates θ₀.

The proof of Proposition 2 is given in Section A.1.

The limit in probability of θ_n has a nice interpretation in terms of projection of Q_Y|A,W (P₀) onto {Q_Y|A,W (θ): θ ∈ Θ}. Preferring to discuss this issue in terms of data generating distribution rather than conditional distribution, let us set $Q_{{\theta }_0}=(Q_W(P_0),Q_{Y|A,W}({\theta }_0))$ and assume that $P_{Q_0,g^r}\text{\hspace{0.17em}log\hspace{0.17em}}{Q}_{Y|Q,W}(P_0)$ is well defined (this weak assumption concerns Q₀, not g^r, and holds for instance when |log Q_Y|A,W (P₀)| is bounded). Then A1 is equivalent to $P_{Q_{{\theta }_0},g^r}$ being the unique Kullback-Leibler projection of $P_{Q_0,g^r}$ onto the set

$$\{P\in \mathcal{M}:\exists \theta \in \Theta \text{\hspace{0.17em}s}.\text{t}.Q_{Y|A,W}(P)=Q_{Y|A,W}(\theta ),\text{\hspace{0.17em}and\hspace{0.17em}}{Q}_W(P)=Q_W(P_0),g(P)=g^r\}.$$

In addition to being consistent, θ_n actually satisfies a central limit theorem if supplementary mild conditions are met. The latter central limit theorem is embedded in a more general result that we state in Section 4.3, see Proposition 5.

Furthermore, maximizing a weighted version of the log-likelihood is a technical twist that makes the theoretical study of the properties of θ_n easier. Indeed, the unweighted maximum likelihood estimator

$$t_n=\underset{\theta \in \Theta }{\text{arg\hspace{0.17em}max}}{\displaystyle \sum _{i=1}^n\text{log\hspace{0.17em}}{Q}_{Y|A,W}(\theta )(O_i)}$$

targets the parameter

$$T_{{\overline{g}}_n}(Q_0)=\underset{\theta \in \Theta }{\text{arg\hspace{0.17em}max}}{\displaystyle \sum _{i=1}^n{P}_{Q_0,g_i^{★}}\text{\hspace{0.17em}log\hspace{0.17em}}{Q}_{Y|A,W}(\theta )=\underset{\theta \in \Theta }{\text{arg\hspace{0.17em}max}}{P}_{Q_0,{\overline{g}}_n}{\mathcal{l}}_{\theta ,0}},$$

where ${\overline{g}}_n=n^{-1}{\displaystyle {\sum }_{i=1}^n{g}_i^{★}}$ and $P_{Q_0,{\overline{g}}_n}f\equiv E_{P_{Q_0,{\overline{g}}_n}}[f(O_n)|{\mathbf{O}}_n(n-1)]$ for any measurable f defined on $\mathcal{O}$. Therefore, t_n asymptotically targets the limit, if it exists, of $T_{{\overline{g}}_n}(Q_0)$. Assuming that ${\overline{g}}_n$ converges itself to a fixed design $g_{\infty }\in \mathcal{G}$, then t_n asymptotically targets parameter $T_{g_{\infty }}\left(Q_0\right)$. The latter parameter is very difficult to interpret and to analyze, as it depends directly and indirectly (through g_∞) on Q₀.

4.2. Convergence of the adaptive design

Consider the mapping G^★ from Θ to ${\mathcal{G}}_1$ (respectively equipped with the Euclidean distance and, for instance, the distance $d(g,g^{\prime })={\displaystyle {\sum }_{v\in \mathcal{V}}|g(1|v)-g^{\prime }(1|v)|)}$ such that, for any θ ∈ Θ, for any $(a,v)\in \mathcal{A}\times \mathcal{V}$,

$$G^{★}(\theta )(a|v)=\frac{{\sigma }_v(a)}{{\sigma }_v(1)+{\sigma }_v(0)}.$$ (13)

Equation (13) characterizes G^★ which is obviously continuous. Since ${\mathbf{g}}_n^{★}$ is adapted in such a way that $g_n^{★}=G^{★}({\theta }_n)$ (see (8)), Proposition 2 and the continuous mapping theorem (see Theorem 1.3.6 in (van der Vaart and Wellner, 1996)) straightforwardly imply the following result (the convergence in probability yields the convergence in L¹ because ${\mathbf{g}}_n^{★}$ is uniformly bounded).

Proposition 3 (convergence of $g_n^{★}$). Under the assumptions of Proposition 2, the adaptive design $g_n^{★}$ converges in probability and in L¹ to the limit design G^★(θ₀).

The convergence of the adaptive design ${\mathbf{g}}_n^{★}$ is a crucial result. It is noteworthy that the limit design G^★(θ₀) equals the optimal design g^★(Q₀) if the working model is correctly specified (which never happens in real-life applications), but not necessarily otherwise. Furthermore, the relationship $g_n^{★}=G^{★}({\theta }_n)$ also entails the possibility to derive the convergence in distribution of $\sqrt{n}(g_n^{★}-G^{★}({\theta }_n))$ to a centered Gaussian distribution with known variance by application of the delta-method (G^★ is differentiable) from a central limit theorem on θ_n (see Proposition 5 and Theorem 3.1 in (van der Vaart, 1998)).

4.3. TMLE

Fluctuating $Q_n^0$.

The second step of the TMLE procedure stretches the initial estimate $\Psi (Q_n^0)$ in the direction of the parameter of interest, through a maximum likelihood step over a well-chosen fluctuation of $Q_n^0$. The latter fluctuation of $Q_n^0$ is just a one-dimensional parametric model $\{Q_n^0(\epsilon ):\epsilon \in \mathcal{E}\}\subset \mathcal{Q}$ indexed by the parameter $\epsilon \in \mathcal{E},\mathcal{E}\subset ℝ$ being a bounded interval which contains a neighborhood of the origin. Specifically, we set for all $\epsilon \in \mathcal{E}$:

$$Q_n^0(\epsilon )=(Q_W(P_n),Q_{Y|A,W}({\theta }_n,\epsilon )),$$ (14)

where for any θ ∈ Θ,

$$Q_{Y|A,W}(\theta ,\epsilon )(O)=\frac{1}{\sqrt{2\pi {\sigma }_V^2(A)}}\text{exp}\left\{-\frac{{(Y-\overline{Q}(\theta )(A,W)-\epsilon H^{★}(\theta )(A,W))}^2}{2{\sigma }_V^2(A)}\right\},$$ (15)

with

$$H^{*}(\theta )(A,W)=\frac{2A-1}{G^{★}(\theta )(A|V)}{\sigma }_V^2(A).$$

In particular, the fluctuation goes through $Q_n^0$ at ε = 0 (i.e., $Q_n^0(0)=Q_n^0$). Let $P_n^0(\epsilon )\in \mathcal{M}$ be a data generating distribution such that $Q_{Y|A,W}(P_n^0(\epsilon ))=Q_{Y|A,W}({\theta }_n,\epsilon )$. The conditional mean $\overline{Q}(P_n^0(\epsilon ))$, which we also denote by $\overline{Q}({\theta }_n,\epsilon )$, is

$$\overline{Q}({\theta }_n,\epsilon )(A,W)=\overline{Q}({\theta }_n)(A,W)+\epsilon H^{*}({\theta }_n)(A,W).$$

Furthermore, the score at ε = 0 of $P_n^0(\epsilon )$ equals

$${\frac{\partial }{\partial \epsilon }\text{log\hspace{0.17em}}{P}_n^0(\epsilon )[O]|}_{\epsilon =0}=\frac{2A-1}{G^{★}({\theta }_n)(A|V)}(Y-\overline{Q}({\theta }_n)(A,W))=D_2^{★}(P_{Q_n^0,G^{★}({\theta }_n)})(O),$$

the second component of the efficient influence curve of Ψ at $P_{Q_n^0,G^{★}({\theta }_n)}=P_{Q_n^0,g_n^{★}}$, see (2) in Proposition 1 (recall that $g_n^{★}=G^{★}({\theta }_n)$).

Characterizing $Q_n^{*}$ yields the TMLE.

We characterize the update $Q_n^{*}$ of $Q_n^0$ in the fluctuation $\{Q_n^0(\epsilon ):\epsilon \in \mathcal{E}\}$ by

$$Q_n^{*}=Q_n^0({\epsilon }_n),$$

where

$${\epsilon }_n=\underset{\epsilon \in \mathcal{E}}{\text{arg\hspace{0.17em}max}}{\displaystyle \sum _{i=1}^n\text{log\hspace{0.17em}}{Q}_{Y|A,W}({\theta }_n,\epsilon )(O_i)\frac{g_n^{★}(A_i|V_i)}{g_i^{★}(A_i|V_i)}}$$ (16)

is a weighted maximum likelihood estimator with respect to the fluctuation. It is worth noting that ε_n is known in closed form (we assume, without serious loss of generality, that $\mathcal{E}$ is large enough for the maximum to be achieved in its interior). Denoting the vth component θ_n(v) of θ_n by $({\beta }_{v,n},{\sigma }_{v,n}^2(0),{\sigma }_{v,n}^2(1))$, it holds that

$${\epsilon }_n=\frac{{\displaystyle {\sum }_{i=1}^n(Y_i-\overline{Q}({\theta }_n)(A_i,W_i))\frac{2A_i-1}{g_i^{★}(A_i|V_i)}}}{{\displaystyle {\sum }_{i=1}^n\frac{{\sigma }_{V_i,n}^2(A_i)}{g_n^{★}(A_i|V_i)g_i(A_i|V_i)}}}.$$

The notation $Q_n^{*}$ for this first update of $Q_n^0$ is a reference to the fact that the TMLE procedure, which is in greater generality an iterative procedure, converges here in one single step. Indeed, say that one fluctuates $Q_n^{*}$ as we fluctuate $Q_n^0$ i.e., by introducing

$$Q_n^1(\epsilon )=(Q_W(P_n),Q_{Y|A,W}({\theta }_n,{\epsilon }_n,\epsilon ))$$

with Q_Y|A,W (θ,ε’,ε) equal to the right-hand side of (15) where one substitutes $\overline{Q}(\theta ,{\epsilon }^{\prime })$ for $\overline{Q}(\theta )$. Say that one then defines the weighted maximum likelihood ${{\epsilon }^{\prime }}_n$ as the right-hand side of (16) where one substitutes Q_Y|A,W (θ_n, ε_n, ε) for Q_Y|A,W (θ_n, ε). Then it follows that ${{\epsilon }^{\prime }}_n=0$ so that the “updated” $Q_n^{*}({{\epsilon }^{\prime }}_n)=Q_n^{*}$. The updated estimator $Q_n^{*}$ of Q₀ maps into the TMLE ${\psi }_n^{*}=\Psi (Q_n^{*})$ of the risk difference ψ₀ = Ψ(Q₀):

$${\psi }_n^{*}=\frac{1}{n}{\displaystyle \sum _{i=1}^n\overline{Q}({\theta }_n,{\epsilon }_n)(1,W_i)-\overline{Q}({\theta }_n,{\epsilon }_n)(0,W_i)}.$$ (17)

The asymptotic study of ${\psi }_n^{*}$ relies on a central limit theorem for (θ_n, ε_n), which we discuss in Section 5.3.

5. Asymptotics

5.1. Studying $Q_n^{*}$ through (θ_n, ε_n): consistency

We now state and comment on a consistency result for the stacked estimator (θ_n, ε_n) which complements Proposition 2 (see Theorem 8 in (van der Laan, 2008)). For simplicity, let us generalize the notation ℓ_θ,₀ introduced in Section 4.1 by setting, for all $(\theta ,\epsilon )\in \Theta \times \mathcal{E}$,

$${\mathcal{l}}_{\theta ,\epsilon }=\text{log\hspace{0.17em}}{Q}_{Y|A,W}(\theta ,\epsilon ).$$

Moreover, let us set, for all $(\theta ,\epsilon )\in \Theta \times \mathcal{E}$,

$$Q_{\theta ,\epsilon }=(Q_W(P_0),Q_{Y|A,W}(\theta ,\epsilon )).$$ (18)

Proposition 4 (consistency of (θ_n,ε_n)). Suppose that A1 and A2 from Proposition 2 hold. In addition, assume that:

A3. There exists a unique interior point ${\epsilon }_0\in \mathcal{E}$ such that ${\epsilon }_0=\text{arg\hspace{0.17em}}{\text{max}}_{\epsilon \in \mathcal{E}}{P}_{Q_0,G^{★}({\theta }_0)}{\mathcal{l}}_{{\theta }_0,\epsilon }$.

• (i)It holds that $\Psi (Q_{{\theta }_0,{\epsilon }_0})=\Psi (Q_0)$.
• (ii)Provided that $\mathcal{O}$ is a bounded set, (θ_n,ε_n) consistently estimates (θ₀,ε₀).

The proof of Proposition 4 is given in Section A.1.

We already discussed the interpretation of the limit in probability of θ_n in terms of Kullback-Leibler projection. Likewise, the limit in probability ε₀ of ε_n enjoys such an interpretation. Let us assume that $P_{Q0,G^{★}({\theta }_0)}$ log Q_Y|A,W(P₀) is well defined (this weak assumption concerns Q₀, not G★(θ₀), and holds for instance when |log Q_Y|A,W(P₀)| is bounded). Then A3 is equivalent to $P_{Q_{{\theta }_0,{\epsilon }_0},G^{★}}({\theta }_0)$ being the unique the Kullback-Leibler projection of $P_{Q_0,G^{★}}({\theta }_0)$ onto the set

$$\{P\in M:\exists \epsilon \in \mathcal{E}\text{\hspace{0.17em}s}.\text{t}.Q(P)=Q_{{\theta }_{0,\epsilon }}\text{\hspace{0.17em}and\hspace{0.17em}}g(P)=G^{★}({\theta }_0)\}.$$

Of course, the most striking property that ε₀ enjoys is (i): even if $\overline{Q}0\notin \{\overline{Q}(\theta ,\epsilon ):(\theta ,\epsilon )\in \Theta \times \mathcal{E}\}$, it holds that $\Psi (Q_{{\theta }_0,{\epsilon }_0})=\Psi (Q_0)$. This remarkable equality and the convergence of (θ_n,ε_n) to (θ₀,ε₀) are evidently the keys to the consistency of ${\psi }_n^{*}=\Psi (Q_n^{*})$. We investigate in Section 5.3 how the consistency result stated in Proposition 4 translates into the consistency of the TMLE.

5.2. Studying $Q_n^{*}$ through (θ_n,ε_n): central limit theorem

We now state and comment on a central limit theorem for the stacked estimator (θ_n, ε_n) (see also Theorem 9 in (van der Laan, 2008)).

Let us introduce, for all $(\theta ,\epsilon )\in \Theta \times \mathcal{E}$,

$$D(\theta ,\in )(O,Z)=\frac{1}{g_Z(A|V)}{\left({\dot{\mathcal{l}}}_{\theta ,0}^{{\rm T}}(O)g^r(A|V),\frac{\partial }{\partial \epsilon }{\mathcal{l}}_{\theta ,\epsilon }(O)G^{★}(\theta )(A|V)\right)}^{⊺},$$ (19)

and $\tilde{D}(\theta ,\epsilon )(O)=g_Z(A|V)D(\theta ,\epsilon )(O,Z)$.

Proposition 5 (central limit theorem for (θ_n,ε_n)). Suppose thatA1, A2andA3from Propositions 2 and 4 hold. In addition, assume thatA4. If a deterministic function F is such that F(O) = 0 $P_{Q_0,G^{★}({\theta }_0)}$-almost surely, then F = 0. Then the following asymptotic linear expansion holds:

$$\sqrt{n}(({\theta }_n,{\epsilon }_n)-({\theta }_0,{\epsilon }_0))=S_0^{-1}\frac{1}{\sqrt{n}}{\displaystyle \sum _{i=1}^{n}D({\theta }_0,{\epsilon }_0)(O_i,Z_i)+o_P(1)},$$ (20)

where

$$S_0=E_{Q_0,G★({\theta }_0)}\left(\begin{array}{cc}{\ddot{\mathcal{l}}}_{{\theta }_0,0}(O)\frac{g^r(A|V)}{G★({\theta }_0)(A|V)}& 0\\ \left[[\frac{{\partial }^2}{\partial \theta \partial \epsilon }{\mathcal{l}}_{\theta ,\epsilon }(O)G★(\theta )(A|V)\right]|{}_{{}_{(\theta ,\epsilon )=({\theta }_0,\epsilon )}}^{\top }\frac{1}{G★({\theta }_0)(A|V)}& \frac{{\partial }^2}{\partial {\epsilon }^2}{\mathcal{l}}_{{\theta }_{0,\epsilon }}(O)|{}_{\epsilon ={\epsilon }_0}\end{array}\right)$$ (21)

is an invertible matrix. Furthermore, (20) entails that $\sqrt{n}(({\theta }_n,{\epsilon }_n)-({\theta }_0,{\epsilon }_0))$ converges in distribution to the centered Gaussian distribution with covariance matrix $S_0^{-1}{\sum }_0{(S_0^{-1})}^{⊺}$, where

$${\Sigma }_0=E_{Q_0,G^{★}({\theta }_0)}\left(\frac{\tilde{D}({\theta }_0,{\epsilon }_0)\tilde{D}{({\theta }_0,{\epsilon }_0)}^{⊺}(O)}{G^{★}({\theta }_0){(A|V)}^2}\right)$$ (22)

is a positive definite symmetric matrix. Moreover, S₀ is consistently estimated by

$$S_n=\frac{1}{n}{\displaystyle \sum _{i=1}^n\left(\begin{array}{cc}{\ddot{\mathcal{l}}}_{{\theta }_n,0}(O_i)\frac{g^r(A_i|V_i)}{g{z}_i(A_i|V_i)}& 0\\ \left[[\frac{{\partial }^2}{\partial \theta \partial \epsilon }{\mathcal{l}}_{\theta ,\epsilon }(O_i)G★(\theta )(A_i|V_i)\right]|{}_{{}_{(\theta ,\epsilon )=({\theta }_n,{\epsilon }_n)}}^{⊺}\frac{1}{g{z}_i★(A_i|V_i)}& \frac{{\partial }^2}{\partial {\epsilon }^2}{\mathcal{l}}_{{\theta }_{n,\epsilon }}(O_i)|{}_{\epsilon ={\epsilon }_n}g{z}_i(A_i|V_i)\end{array}\right)}$$

and ∑₀ is consistently estimated by

$${\sum }_n=\frac{1}{n}{\displaystyle \sum _{i=1}^{n}D({\theta }_n,{\epsilon }_n)D{({\theta }_n,{\epsilon }_n)}^{\top }(O_i,Z_i).}$$ (23)

The proof of Proposition 5 is given in Section A.2. We investigate in Section 5.3 how the above central limit theorem translates into a central limit theorem for the TMLE.

5.3. Consistency and asymptotic normality of the TMLE

In this section, we finally state and comment on the asymptotic properties of the TMLE ${\psi }_n^{*}$.

TMLE is consistent and asymptotically Gaussian.

In the first place, the TMLE ${\psi }_n^{*}$ is robust: it is a consistent estimator even when the working model is misspecified (which is always the case in real-life applications).

Proposition 6 (consistency of ${\psi }_n^{*}$). Suppose that A1, A2, A3 and A4 from Propositions 2, 4 and 5 hold. Then the TMLE ${\psi }_n^{*}$ consistently estimates the risk difference ψ₀.

If the design of the clinical trial was fixed (and consequently, the n first observations were i.i.d), then the TMLE would be a robust estimator of ψ₀: Even if the working model is misspecified, then the TMLE still consistently estimates ψ₀ because the treatment mechanism is known (or can be consistently estimated, if one wants to gain in efficiency). Thus, the robustness of the TMLE stated in Proposition 6 is the expected counterpart of the TMLE’s robustness in the latter i.i.d setting: Expected because the TMLE solves a martingale estimating function that is unbiased for ψ₀ at misspecified Q and correctly specified g_i, i = 1,…,n.

In the second place, the TMLE ${\psi }_n^{*}$ is asymptotically linear, and therefore satisfies a central limit theorem. To see this, let us introduce the real-valued function ϕ on $\Theta \times \mathcal{E}$ such that ϕ (θ, ∈) = Ψ(Q_θ,ε) (see (18) for the definition of Q_θ,ε). The function ϕ is differentiable on the interior of $\Theta \times \mathcal{E}$, and we denote ${{\varphi }^{\prime }}_{\theta ,\epsilon }$ its gradient at (θ,ε). The latter gradient satisfies

$${\varphi }_{\theta ,\epsilon }^{\prime }=E_{Q_{\theta ,\epsilon },G^{★}(\theta )}\left\{D^{★}(P_{Q_{\theta ,\epsilon }},G^{★}(\theta )(O){\left(\frac{\partial }{\partial \theta }{\mathcal{l}}_{\theta ,\epsilon }^{\top }(O),\frac{\partial }{\partial \epsilon }{\mathcal{l}}_{\theta ,\epsilon }(O)\right)}^{\top }\right\}.$$ (24)

Note that the right-hand side expression cannot be computed explicitly because the marginal distribution Q_W(P₀) is unknown. By the law of large numbers (independent case), we can build an estimator ${{\varphi }^{\prime }}_n$ of ${{\varphi }^{\prime }}_{{\theta }_0,{\epsilon }_0}$ as follows. For B a large number (say B = 10⁴), simulate B independent copies Õ_b of O from the data generating distribution $P_{Q_n^{*},G^{★}({\theta }_n)}$, then compute

$${\varphi }_n^{\prime }=\frac{1}{B}{\displaystyle \sum _{b=1}^B{D}^{★}}\left(P_{Q_n^{*},G^{★}({\theta }_n)}\right)(O_b){\left(\frac{\partial }{\partial \theta }{\mathcal{l}}_{\theta ,{\epsilon }_n}^{\top }(O_b){|}_{\theta ={\theta }_n},\frac{\partial }{\partial \epsilon }{\mathcal{l}}_{{\theta }_n,\epsilon }(O_b)|\epsilon ={\epsilon }_n\right)}^{\top }.$$ (25)

Proposition 7 (central limit theorem for ${\psi }_n^{*}$). Suppose that A1, A2, A3 and A4 from Propositions 2, 4 and 5 hold. Then the following asymptotic linear expansion holds:

$$\sqrt{n}({\psi }_n^{*}-{\psi }_0)=\frac{1}{\sqrt{n}}{\displaystyle \sum _{i=1}^n\text{IC\hspace{0.17em}}(O_i,Z_i)+oP(1),}$$ (26)

where

$$\text{IC}(O,Z)=D_1^{★}(Q_{{\theta }_0,{\epsilon }_0})(W)+{\varphi }_{{\theta }_0,{\epsilon }_0}^{\prime \top }{S}_0^{-1}D({\theta }_0,{\epsilon }_0)(O,Z).$$ (27)

Furthermore, (26) entails that $\sqrt{n}({\psi }_n^{*}-{\psi }_0)$ converges in distribution to the centered Gaussian distribution with a variance consistently estimated by

$$s_n^2=\frac{1}{n}{\displaystyle \sum _{i=1}^n{D}_1^{★}(Q_n^{*}){(W_i)}^2}+\frac{2}{n}{\displaystyle \sum _{i=1}^n{D}_1^{★}(Q_n^{*})(W_i)}{\varphi }_n^{\prime \top }{S}_n^{-1}D({\theta }_n,{\epsilon }_n)(O_i,Z_i)+({\varphi }_n^{\prime \top }{S}_n^{-1}){\sum }_n{({\varphi }_n^{\prime \top }{S}_n^{-1})}^{\top }.$$

Proposition 7 is the backbone of the statistical analysis of adaptive group sequential RCTs as constructed in Section 4. In particular, denoting the (1 − α)-quantile of the standard normal distribution by ξ_1−α, the proposition guarantees that the asymptotic level of the confidence interval

$$\left[{\psi }_n^{*}\pm \frac{s_n}{\sqrt{n}}{\xi }_{1-\alpha /2}\right]$$ (28)

for the risk difference ψ₀ is (1 − ε). The proofs of Propositions 6 and 7 are given in Section A.3.

Extensions.

We conjecture that the influence function IC computed at (O, Z), see (27), is equal to

$$D_1^{★}(Q_{{\theta }_0,{\epsilon }_0})(W)+D_2^{★}(P_{Q_{{\theta }_0,{\epsilon }_0}}{}_{,G★({\theta }_0)})(O)\frac{G★({\theta }_0)(A|V)}{gZ(A|V)}.$$

This conjecture is backed by the simulations that we carry out and present in Section 7. We will tackle the proof of the conjecture in future work. Let us assume for the moment that the conjecture is true. Then the asymptotic linear expansion (26) now implies that the asymptotic variance of $\sqrt{n}({\psi }_n^{*}-{\psi }_0)$ can be consistently estimated by

$$s_n^{★2}=\frac{1}{n}{\displaystyle \sum _{i=1}^n{\left(D_1^{★}(Q_n^{*})(W_i)+D_2^{★}(P_{Q_n^{*},G^{★}({\theta }_n)})(O_i)\frac{G*({\theta }_n)(A_i|V_i)}{g{Z}_i(A_i|V_i)}\right)}^2,}$$

another independent argument also showing that $s_n^{★2}$ converges toward

$${\text{Var}}_{Q_0,G^{★}({\theta }_0)}{D}^{★}(P_{Q_{{\theta }_0,{\epsilon }_0},G^{★}({\theta }_0)})(O)$$

i.·e.·, the variance under the fixed design $P_{Q_0,G^{★}({\theta }_0)}$ of the efficient influence curve at $P_{Q_{{\theta }_0,{\epsilon }_0},G^{★}({\theta }_0)}$.

Furthermore, the most essential characteristic of the joint methodologies of design adaptation and targeted maximum likelihood estimation is certainly the utmost importance of the role played by the likelihood. In this view, the targeted maximized log-likelihood of the data

$${\displaystyle \sum _{i=1}^n\{\text{log\hspace{0.17em}}{Q}_W(P_n)(W_i)+\text{\hspace{0.17em}log\hspace{0.17em}}{g}_i^{★}(A_i|W_i)+\text{\hspace{0.17em}log\hspace{0.17em}}{Q}_{Y|A,W}({\theta }_n,{\epsilon }_n)(O_i)}\}$$

provides us with a quantitative measure of the quality of the fit (targeted toward the parameter of interest). It is therefore possible, for example, to use that quantity for the sake of selection of different working models for Q₀. As with TMLE for i.i.d data, we can use likelihood based crossvalidation to select among more general initial estimators indexed by fine-tuning parameters. The validity of such TMLEs for group sequential adaptive designs as studied here is outside the scope of this article.

6. Application to group sequential testing

We derive in this section a group sequential testing procedure, that is a testing procedure which repeatedly tries to make a decision at intervals rather than once all data are collected, or than after every new observation is obtained (such a testing procedure would be said fully sequential). We refer to (Jennison and Turnbull, 2000; Proschan et al., 2006) for a general presentation of group sequential testing procedures. The TMLE group sequential testing procedure is formally described in Section 6.1, and some arguments justifying why it should work well (as validated by the simulation study carried out in Section 8) are exposed in Section 6.2.

6.1. Description of the TMLE group sequential testing procedure

The problem at stake is to test the null “Ψ(Q) = ψ₀” against “Ψ(Q) > ψ₀” with asymptotic type I error α and asymptotic type II error β at some ψ₁ > ψ₀. We want to proceed group sequentially with K ≥ 2 steps, based on the multi-dimensional t-statistic

$$(T_1^{*},\dots ,T_K^{*})={\left(\frac{\sqrt{N_k}({\psi }_{N_k}^{*}-{\psi }_0}{s_{N_k}}\right)}_{k\le K}.$$ (29)

Here, N₁,…,N_K are random sample sizes whose realizations on a specific trajectory depend on how fast the information accrues as the data are collected. In order to quantify this notion of information, we decide to consider the inverse $\frac{n}{s_n^2}$ of the estimated variance of the TMLE ${\psi }_n^{*}$ based on the n first observations O_n (as a proxy to its true, finite sample, inverse variance). Given a reference maximum committed information I_max and K increasingly order proportions 0 < p1 < … < p_K = 1, we set for every k ≤ K

$$N_k=\text{\hspace{0.17em}inf\hspace{0.17em}}\left\{n\ge 1:\frac{n}{s_n^2}\ge p_k{\mathrm{I}}_{\text{max}}\right\}.$$

The characterization of I_max depends on how we wish to “spend” the type I and type II errors at each step of the group sequential procedure and on how demanding is the power requirement (i.e., how close ψ₁ is to ψ₀). Say that our spending strategies are summarized by the K-tuples of positive numbers (α₁,…,α_K) and (β₁,…,β_K) such that ${\displaystyle {\sum }_{k=1}^K{\alpha }_k=\alpha }$ and ${\displaystyle {\sum }_{k=1}^K{\beta }_k=\beta }$.

Now, let (Z₁,…,Z_K) be distributed from the centered Gaussian distribution with covariance matrix $\mathcal{C}={(\sqrt{p_{k\wedge l}/p_{k\vee l}})}_{k,l\le K}$ We assume that there exists a unique value I > 0, our I_max, such that there exist a rejection boundary (a₁,…,a_K) and a futility boundary (b₁,…,b_K) satisfying a_K = b_K, P(Z₁ ≥ a₁) = α₁, $P(Z_1+({\psi }_1-{\psi }_0)\sqrt{p_{1}I}\le b_1)={\beta }_1$, and for every 1 ≤ k < K,

$$P(\forall j\le k,b_j<Z_j<a_j\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}and\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}{Z}_{k+1}\ge a_{k+1})={\alpha }_{k+1},$$

$$P(\forall j\le k,b_j<Z_j+({\psi }_1-{\psi }_0)\sqrt{p_{j}I}<a_j\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}and\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}{Z}_{k+1}+({\psi }_1-{\psi }_0)\sqrt{p_{k+1}I}\le b_{k+1})={\beta }_{k+1}.$$

Note that the closer ψ₁ is to ψ₀, the larger is I_max (actually, ${\psi }_1\mapsto {\psi }_1\sqrt{I_{\text{max}}}$ is both upper bounded and bounded away from zero). Heuristically, the closer ψ₁ is to ψ₀, the more difficult it is to decide between the null and its alternative while preserving the required type II error at ψ₁, the more information is needed to proceed.

The targeted maximum likelihood group sequential testing procedure finally goes as follows: starting from k = 1,

• if $T_k^{*}\ge a_k$ then reject the null and stop accruing data,

• if $T_k^{*}\le b_k$ then fail rejecting the null and stop accruing data,

• if $b_k<T_k^{*}<a_k$ then set k ← k + 1 and repeat.

If ($T_1^{*},\dots ,T_K^{*}$) had the same distribution as (Z₁,…,Z_K), then the latter rule would yield a testing procedure with the required type I error and type II error at the specified alternative parameter.

Clearly, our decision to target the optimal design G★(θ₀) which reduces as much as possible (over ${\mathcal{G}}_1$ and for our choice of working model) the asymptotic variance of the TMLE ${\psi }_n^{*}$ guarantees, at least informally, that each N_k is stochastically smaller under (Q₀, ${\mathbf{g}}_n^{★}$)-adaptive sampling than it would have been had another fixed design been used (or targeted). Thus, resorting to the (Q₀,$g_n^{★}$)-adaptive sampling is likely to result in an earlier conclusion than what would have yielded another fixed (or targeted) sampling scheme.

6.2. Rationale of the TMLE group sequential testing procedure

The next proposition partially justifies the characterization of the TMLE group sequential testing procedure of Section 6.1. First, let us define $n_k=\lceil n{p}_k\rceil$ (the smallest integer not smaller than np_k) for every k ≤ K, then

$$(T_1,\dots ,T_K)={\left(\frac{\sqrt{n_k}({\psi }_{nk}^{*}-{\psi }_0}{s_{n_k}}\right)}_{k\le K}.$$

This other multi-dimensional t-statistic is a substitute to $(T_1^{*},\mathrm{...},T_K^{*})$ whose asymptotic study is easier to carry out. In particular, it is possible to derive the limit distribution of (T₁,…,T_K) under the null and under a sequence of contiguous alternatives. The limit distribution is called the canonical distribution (see Theorems 11 in 12 in (van der Laan, 2008), Theorem 3 in (Chambaz and van der Laan, 2011) and Theorem 2.1 in (Zhu and Hu, 2010), where a similar result is obtained through a different approach based on the study of the limit distribution of a stochastic process defined over (0, 1]).

Proposition 8. Suppose that A1, A2, A3 and A4 from Propositions 2, 4 and 5 hold. Consider $f\in L^2(P_{Q_{0,}{G}^{★}({\theta }_0)})$, f ≠ 0, with $P_{Q_{0,}{G}^{★}({\theta }_0)}f=0$ and a fluctuation $\left\{Q_0(h):h\in \mathcal{H}\right\}\subset \mathcal{Q}$ of Q₀ with score f. Set $Q_{f/\sqrt{n}}=Q_0(1/\sqrt{n})$ for all n ≥n₀ (n₀ is such that $1/\sqrt{n_0}\in \mathscr{H}$). Assume also that A5. The score f is bounded, it is not proportional to $D^{★}(P_{Q_0,G^{★}({\theta }_0)})$, $E_{P_{Q_0,G^{★}({\theta }_0)}}[f(O)|A,W]=0,P_{Q_0,G^{★}({\theta }_0)}{D}^{★}(P_{Q_0,G^{★}({\theta }_0)})f>0$, and $P_{Q_0,G^{★}({\theta }_0)}\text{IC}f>0$.

The sequence $(Q_{f/\sqrt{n}})n\ge n_0$ defines a sequence (ψ_n)n≥n₀ of contiguous parameters (“from direction f”, which only fluctuates the conditional distribution of Y given (A, W)), with ${\psi }_n=\text{ψ(}{Q}_{f/\sqrt{n}}\text{)>ψ(}{Q}_0\text{)}$ for n large enough. Introduce $\mu \text{(}f\text{)=(}\sqrt{p_1}\text{,}\mathrm{...}\text{,}\sqrt{p_K}\text{)}\frac{P_{Q_0,G^{★}({\theta }_0)}\text{IC}f}{\sqrt{P_{Q_0,G^{★}({\theta }_0)}{\text{IC}}^{\text{2}}}}.$

• (i)Under (Q₀, g_n)-adaptive sampling, (T₁,…,T_K) converges in distribution to the centered Gaussian distribution with covariance matrix $\mathcal{C}$.
• (ii)Under $\left(Q_{f/\sqrt{n}},g_n\right)$-adaptive sampling, (T₁,…,T_K) converges in distribution to the Gaussian distribution with mean μ(f) and covariance matrix $\mathcal{C}$.

The proof of Proposition 8 is given in Section A.4.

The rationale follows. Say that one concludes from Proposition 8 that (i) ($T_1^{★},\dots ,T_K^{★}$) is also approximately distributed as (Z₁,…,Z_K) under $(Q_0,g_n^{★})$-adaptive sampling. Likewise, say that $(\sqrt{N_k}({\psi }_{N_k}^{*}-{\psi }_1)/s_{N_k})k\le K$ is also approximately distributed as (Z₁,…,Z_K) under $(Q_1,g_n^{★})$-adaptive sampling, with Ψ(Q₁) = ψ₁ > ψ₀. Say that one is willing to substitute p_kI_max to $N_k/s_{N_k}^2$ for each k ≤ K. Then (ii) ($T_1^{*},\dots ,T_K^{*}$) is approximately distributed as $\sqrt{I_{\text{max}}}({\psi }_1-{\psi }_0)(\sqrt{p_1},\mathrm{...},\sqrt{p_K})+(Z_1,\mathrm{...},Z_K)$ under $(Q_1,g_n^{★})$-adaptive sampling. It appears that (i) and (ii) suffice to guarantee the wished asymptotic control of type I and type II errors. The rationale is validated by the simulation study undertaken in Section 8.

7. Simulation study of the performances of TMLE in adaptive covariate-adjusted RCTs

In this section, we carry out a simulation study of the performances of TMLE in adaptive group sequential RCTs as exposed in the previous sections. We present the simulation scheme in Section 7.1. The working model upon which the TMLE methodology relies is described in Section 7.2. How the TMLE-based confidence intervals behave is considered in Section 7.3 (empirical coverage) and in Section 7.4 (empirical width). An illustrating example is finally presented in Section 7.5.

7.1. Simulation scheme

We characterize the component Q₀ = Q(P₀) of the true distribution P₀ of the observed data structure O = (W, A, Y) as follows:

• the baseline covariate W = (U, V) where U is uniformly distributed over the unit interval [0; 1] and the subgroup membership covariate V ∈ v = {1, 2, 3} (hence ν = 3) satisfies

  $$\begin{array}{l}{P}_0(V=1)=\frac{1}{2},P_0(V=2)=\frac{1}{3}\text{\hspace{0.17em}\hspace{0.17em}and\hspace{0.17em}}{P}_0(V=3)=\frac{1}{6};\\ \end{array}$$
• the conditional distribution of Y given (A, W) is the Gamma distribution characterized by the conditional mean

  $${\overline{Q}}_0(A,W)=2U^{2}2U+1+\rho \left(AV+\frac{1-A}{1+V}\right)$$ (30)

  with ρ =1 (we will set ρ to another value in Section 8) and the conditional variance

$$Va{r}_{P_0}(Y|A,W)={\left(U+A(1+V)+\frac{1-A}{1+V}\right)}^2.$$

In particular, the risk difference ψ₀ = ψ(Q₀), our parameter of interest, is known in closed form:

$${\psi }_0=\frac{91}{72}\simeq 1.264,$$

and so is the variance

$$v^b(Q_0)={\text{VarQ}}_{0,g^b}{D}^{★}(O;P_{Q_{0,g^b}})$$

of the efficient influence curve under balanced sampling. The numerical value of v^b(Q₀) is reported in Table 1.

Table 1:

Numerical values of the allocation probabilities and variance of the efficient influence curve under either the balanced g^b or the targeted optimal g^★(Q₀) sampling schemes. The ratio of the variances of the efficient influence curve under targeted optimal and balanced sampling schemes satisfies R(Q₀) = v*(Q₀)/v^b(Q₀) ≃ 0.762.

Sampling scheme (Q0,g)	Allocation probabilities			Variance
	g(1|v = 1)	g(1|v = 1)	g(1|v = 1)	${\text{Var}}_{Q_{0,g}}D*(O;P_{Q_{0,g}})$
(Q0,gb)-balanced	$\frac{1}{2}$	$\frac{1}{2}$	$\frac{1}{2}$	23.864
(Q0,g★ (Q0))-optimal	0.707	0.799	0.849	18.181

We target the design which (a) depends on the baseline covariate W = (U, V) only through V (i.e., belongs to ${\mathcal{G}}_1$) and (b) minimizes the variance of the efficient influence curve of the parameter of interest ψ. The latter treatment mechanism g^★(Q₀) and optimal efficient asymptotic variance

$$v^{★}(Q_0)={\text{Var}}_{Q_{0,g★}(Q_0)}{D}^{★}(O;P_{Q_{0,g★}(Q_0)})$$

are also known in closed form, and numerical values are reported in Table 1.

Let n = (100, 250, 500, 750,1000, 2500, 5000) be a sequence of sample sizes. We estimate M = 1000 times the risk difference ψ₀ = ψ(Q₀) based on $O_{n\tau }^m(n_i)$,m = 1,..., M, i = 1,..., 7, under

• i.i.d (Q₀, g^b)-balanced sampling,

• i.i.d (Q₀, g^★(Q₀))-optimal sampling,

• $(Q_0,g_{n\tau }^{★})$-adaptive sampling.

Finally, we emphasize that the observed data structure O = (W, A, Y) is not bounded, whereas O is assumed bounded in Propositions 2, 3, 4, 5, 6 and 7.

7.2. Specifying the working model, reference design and a few twists On the working model.

For each v ∈ $\mathcal{V}$, let us denote $\theta (\upsilon )=({\beta }_{\upsilon },{\sigma }_{\upsilon }^2(0),{\sigma }_{\upsilon }^2(1))\in {\Theta }_{\upsilon },{\Theta }_{\upsilon }\subset {ℝ}^3\times {ℝ}_{+}^{*}\times {ℝ}_{+}^{*}$ being compact, where the regression vector β_v = (β_v,₁, β_v,₂,β_v,₃) (b = 3 in (10)), then θ = (θ₁,θ₂,θ₃) ∈ Θ = Θ₁ × Θ₂ × Θ₃.

Following the description in Section 4.1, the working model ${\mathcal{Q}}_n^w$ that the TMLE methodology relies on is characterized by the conditional likelihood of Y given (A, W):

$$Q_{Y|A,W}(\theta )(O)=\frac{1}{\sqrt{2\pi {\sigma }_V^2(A)}}\text{exp\hspace{0.17em}\hspace{0.17em}}\left\{-\frac{{(Y-m(A,W;{\beta }_V))}^2}{2{\sigma }_V^2(A)}\right\},$$

with the specific choice of conditional mean $\overline{Q}(\theta )(A,W)$ of Y given (A, W):

$$\begin{array}{l}\overline{Q}(\theta )(\alpha ,w)=m(a,w;{\beta }_v)={\beta }_{v,1}+{\beta }_{v,2}u+{\beta }_{v,3}a\\ \end{array}$$

for all $a\in \mathcal{A}$ and $w=\left(u,v\right)\in \mathcal{W}=ℝ\times \mathcal{V}$. As required, condition PARAM is met. Obviously, the working model is heavily misspecified:

• a Gaussian conditional likelihood is used instead of a Gamma conditional likelihood,

• the parametric forms of the conditional expectation and variance are wrong too.

On the reference design.

Regarding the choice of a reference fixed design g^r ∈ ${\mathcal{G}}_1$ (see Section 4.1), we select g^r = g^b (the balanced design). The parameter θ₀. only depends on Q₀ and the working model, but its estimator θ_n depends on g^r, which may affect negatively its performances. Therefore, we propose to dilute the impact of the choice of g^r as an initial reference design as follows.

For a given sample size n, we first compute a first estimate ${\theta }_n^1$ of θ₀ as in (12) but with $\lceil n/4\rceil$ (the smallest integer not smaller than n/4) substituted for n in the sum. Then θ_n is computed as in (12) but this time with $G*({\theta }_n^1)(A_i|V_i)$ substituted for g^r(A_i|V_i).

The proofs can be adapted in order to incorporate this modification of the procedure. We refer the interested reader to Section 8.5 in (van der Laan, 2008).

On additional details.

We decide arbitrarily to update the design each time c = 25 new observations are sampled. In addition, the first update only occurs when there are at least five completed observations in each treatment arm and for all V-strata. Thus, the minimal sample size at first update is 30. It can be shown that, under initialization using the balanced design, the expected sample size at the first sample size at which there are at least 5 observations in each arm equals 75. Finally, as a precautionary measure, we systematically apply a thresholding to the updated treatment mechanism: Using the notation of Section 4, $\text{max}\{\delta ,min\{1-\delta ,g_i^{*}(A_i|V_i)\}\}$ is substituted for $g_i^{*}(A_i|V_i)$ in all computations. We arbitrarily choose δ = 0.01.

7.3. Empirical coverage of the TMLE confidence intervals

We now invoke the central limit theorem stated in Proposition 7 to construct confidence intervals for the risk difference. Let us introduce, for all types of sampling and each sample size n_i, the confidence intervals

$$\begin{array}{l}{\mathcal{I}}_{n_i,m}=\left[{\psi }_{n_i}^{*}(O_{n_7}^m(n_i))\pm \sqrt{\frac{v_{n_i}(O_{n_7}^m(n_i))}{n_i}}{\xi }_{1-\alpha /2}\right],m=1,\dots ,M\\ \end{array}$$

where the definition of the variance estimator ${\upsilon }_n(O_{n\tau }^m(n))$ based on the n first observations $O_{n\tau }^m(n)$ depends on the sampling scheme:

• under i.i.d (Q₀, g^b)-balanced sampling, ${\upsilon }_n(O_{n\tau }^m(n))$ is the estimator of the asymptotic variance of the TMLE $\text{ψ(}{Q}_{n,iid}^{*}\text{)}$:

$$v_n(O_{n_7}^m(n))=\frac{1}{n}{\displaystyle \sum _{i=1}^n{D}^{★}(P_{Q_{n,\text{iid}}^{*}{g}^b}){(O_i^m)}^2,}$$ (31)

• under i.i.d (Q₀, g^★(Q₀))-optimal sampling, ${\upsilon }_n(O_{n\tau }^m(n))$ is defined as in (31), replacing g^b with g^★(Q₀),

• under $(Q_0,g_n^{★})$-adaptive sampling, ${\upsilon }_n(O_{n\tau }^m(n))=s_n^{*2}(O_{n\tau }^m(n))$ , the estimator of the conjectured asymptotic variance of $\sqrt{n}({\psi }_n^{*}(O_{n\tau }^m(n))-{\psi }_0)$ computed on the n first observations $O_{n\tau }^m(n)$.

We are interested in the empirical coverage (reported in Table 2, top rows)

$$c_{n_i}=\frac{1}{M}{\displaystyle \sum _{m=1}^{M}1\{}{\psi }_0\in {\mathcal{I}}_{n_i,m}\}$$

guaranteed for each sampling scheme and every i = 1,..., 7 by $\{{\mathcal{I}}_{ni,m}:m=1,\mathrm{...},M\}$. The rescaled empirical coverage proportions $M_c{}_{{}_{n_i}}$ should have a Binomial distribution with parameter (M, 1 – a) with a = α for every i = 1,..., 7. This property can be tested in terms of a standard binomial test, the alternative stating that a > α. This results in a collection of 7 p-values for each sampling scheme, as reported in Table 2 (bottom rows).

Table 2:

Checking the adequateness of the coverage guaranteed by our simulated confidence intervals. We test if the rescaled empirical coverage Binomial random variables $M_c{}_{{}_{n_i}}$ have parameter (M, 1 – α), the alternative stating that they have parameter (M, 1 – a) with a > α. We report the values $c_{n_i}$ (top row) and corresponding p-values (bottom row, between parentheses) of the Binomial test for each sample size and each sampling scheme.

Sampling scheme	Sample size
	n1	n2	n3	n4	n5	n6	n7
i.i.d (Q0, gb)-balanced	0.913 (p < 0.001)	0.925 (p < 0.001)	0.939 (0.067)	0.934 (0.015)	0.945 (0.253)	0.940 (0.087)	0.946 (0.300)
i.i.d (Q0, g★(Q0))-optimal	0.894 (p < 0.001)	0.941 (0.111)	0.940 (0.087)	0.953 (0.688)	0.954 (0.739)	0.947 (0.351)	0.947 (0.351)
adaptive $(Q_0,g_{n_7}^{★})$	0.934 (0.015)	0.939 (0.067)	0.956 (0.827)	0.945 (0.253)	0.943 (0.172)	0.933 (0.011)	0.952 (0.634)

Considering each sampling scheme (i.e., each row of Table 2) separately, we conclude that the (1 – α)-coverage cannot be declared defective under

• i.i.d (Q₀, g^b)-balanced sampling for any sample size n_i ≥ n₃ = 500,

• i.i.d (Q₀, g^★(Q₀))-optimal sampling for any sample size n_i ≥ n₂ = 250,

• $(Q_0,g_{n_7}^{★})$-adaptive sampling for any sample size n_i ≥ n₁ = 100,

adjusting for multiple testing in terms of the Benjamini and Yekutieli procedure for controlling the False Discovery Rate at level 5%.

This is a remarkable result that not only validates the theory but also provides us with insight into the finite sample properties of the TMLE procedure based on adaptive sampling. The fact that the TMLE procedure behaves better under adaptive sampling scheme than under balanced i.i.d sampling scheme at sample size n₁ = 100 may not be due to mere chance only. Although the TMLE procedure based on an adaptive sampling scheme is initiated under the balanced sampling scheme (so that each stratum consists at the beginning of comparable numbers of patients assigned to each treatment arm, allowing to estimate, at least roughly, the required parameters), it starts deviating from it (as soon as every (A, V)-stratum counts 5 patients) each time 25 new observations are accrued. The poor performance of the TMLE procedure based on optimal i.i.d sampling scheme at sample size n₁ is certainly due to the fact that, by starting directly from the optimal sampling scheme (a choice we would not recommend in practice), too few patients from stratum V = 3 are assigned to treatment arm A = 0 among the n1 first subjects. At larger sample sizes, the TMLE procedure performs equally well under adaptive sampling scheme and under both i.i.d schemes in terms of coverage.

7.4. Empirical width of the TMLE confidence intervals

Now that we know that the TMLE-based confidence intervals based on $(Q_0,g_n^{★})$-adaptive sampling are valid confidence regions, it is of interest to compare the widths of the latter $(Q_0,g_n^{★})$-adaptive sampling confidence intervals with their counterparts obtained under i.i.d (Q₀, g^b)-balanced or (Q₀,g*(Q₀))-optimal sampling schemes.

For this purpose, we compare on one hand, for each sample size n_i, the empirical distribution of $\{\sqrt{{\upsilon }_n(O_{n\tau }^m(n_i))}:m=1,\mathrm{...},M\}$ as in (31) (i.e., the empirical distribution of width of the TMLE-based confidence intervals at sample size n_i obtained under i.i.d (Q₀, g^b)-balanced sampling, up to the factor $2{\xi }_{1-\alpha /2}/\sqrt{n_i}$) to the empirical distribution of $\{s_n^{★}(O_{n\tau }^m(n_i)):m=1,\mathrm{...},M\}$ (i.e., the empirical distribution of the width of the TMLE-based confidence intervals at sample size n_i obtained under $(Q_0,g_n^{★})$ adaptive sampling, up to the factor $2{\xi }_{1-\alpha /2}/\sqrt{n_i}$ in terms of the two-sample Kolmogorov-Smirnov test, where the alternative states that the confidence intervals obtained under adaptive sampling are stochastically smaller than their counterparts under i.i.d balanced sampling. This results in 7 p-values, all equal to zero, which we nonetheless report in Table 3 (bottom row). In order to get a sense of how much narrower the confidence intervals obtained under adaptive sampling are, we also compute and report in Table 3 (top row) the ratios of empirical average widths

$$\frac{\frac{1}{M}{\Sigma }_{m=1}^M{s}_n^{★}(O_{n\tau }^m(n_i))}{\frac{1}{M}{\Sigma }_{m=1}^M\sqrt{v_n(O_{n\tau }^m(n_i))}}$$ (32)

for each sample size n_i. Informally, this shows a 12% gain in width.

Table 3:

Comparing the width of our confidence intervals. On one hand, we test for each sample size n_i if the TMLE-based confidence intervals obtained under $(Q_0,g_n^{★})$-adaptive sampling are narrower stochastically than the TMLE-based confidence intervals obtained under i.i.d (Q₀, g^b)-balanced sampling in terms of the two-sample Kolmogorov-Smirnov test. On the other hand, we test for each sample size n_i if the TMLE-based confidence intervals obtained under $(Q_0,g_n^{★})$-adaptive sampling are wider stochastically than the TMLE-based confidence intervals obtained under i.i.d (Q₀,g*(Q₀))-optimal sampling in terms of the two-sample Kolmogorov-Smirnov test. We report the p-values (bottom rows, between parentheses). In addition, we report for each sample size n_i the ratios of average widths as defined in (32).

Comparison	Sample size
	n1	n2	n3	n4	n5	n6	n7
$(Q_0,g_{n_7}^{★})vs(Q_0,g^b)$	0.856 (0)	0.871 (0)	0.879 (0)	0.880 (0)	0.878 (0)	0.877 (0)	0.876 (0)
$(Q_0,g_{n_7}^{★})vs(Q_0,g^{★}(Q_0))$	0.962 (0.144)	0.977 (0.236)	0.992 (0.100)	0.995 (0.060)	0.997 (0.407)	1.000 (0.236)	1.000 (0.144)

On the other hand, we also compare, for each sample size n_i, the empirical distribution of $\{\sqrt{{\upsilon }_n(O_{n\tau }^m(n_i))}:m=1,\mathrm{...},M\}$ as in (31) but replacing g^b by g*(Q₀) (i.e., the empirical distribution of width of the TMLE-based confidence intervals at sample size n_i obtained under i.i.d (Q₀, g*(Q₀))-optimal sampling, up to the factor $2{\xi }_{1-\alpha /2}/\sqrt{n_i}$ to the empirical distribution of $\{s_n^{*}(O_{n\tau }^m(n_i)):m=1,\mathrm{...},M\}$ (i.e., the empirical distribution of the width of the TMLE-based confidence intervals at sample size n_i obtained under $(Q_0,g_n^{★})$-adaptive sampling, up to the factor $2{\xi }_{1-\alpha /2}/\sqrt{n_i}$ in terms of the two-sample Kolmogorov-Smirnov test, where the alternative states that the confidence intervals obtained under adaptive sampling are stochastically larger than their counterparts under i.i.d optimal sampling. This results in 7 p-values that we report in Table 3 (bottom row). In order to get a sense of how similar the confidence intervals obtained under adaptive and i.i.d optimal sampling schemes are, we also compute and report for each sample size n_i in Table 3 (top row) the ratios of empirical average widths as in (32) replacing again g^b by g*(Q₀) in the definition (31) of ${\upsilon }_n(O_{n\tau }^m(n))$. Informally, this shows that the confidence intervals obtained under adaptive sampling are even slightly narrower in average than their counterparts obtained under i.i.d optimal sampling.

7.5. Illustrating example

So far we have been concerned with distributional results, answering to the questions Does the confidence interval $[{\psi }_n^{*}\pm s_n^{*}{\xi }_{1-\alpha /2}/\sqrt{n}]$ provide the wished coverage? (yes, even for moderate sample size: see Section 7.3), How does its width compare with the width of the confidence interval obtained under either i.i.d sampling scheme? (well: see Section 7.4). In this section, we focus on a particular simulated trajectory (we select arbitrarily the first one, associated to $O_{n_7}^1$) for the sake of illustration.

Some interesting features of the selected simulated trajectory are apparent in Fig. 7.5 and Table 4.

Table 4:

Illustrating the TMLE procedure under $(Q_0,g_n^{★})$-adaptive sampling scheme. This table illustrates how the TMLE procedure behaves (on a simulated trajectory) as the sample size increases. We report, at each sample size n_i, the updated adapted design $g_n^{★}=G^{★}({\theta }_n(O_{n_7}^1(n)))$ (columns two to four), current TMLE ${\psi }_n^{*}(O_{n_7}^1(n))$ (column five) and 95% confidence interval I_n,1 as in (28) with $s_n^{*}(O_{n_7}^1(n))$ substituted to s_n. The true risk difference ψ₀ = ≃ 1.264 belongs to all confidence intervals. See also Fig. 7.5.

Sample size	Allocation probabilities			TMLE	Confidence interval
n	$g_n^{★}(1|1)$	$g_n^{★}(1|2)$	$g_n^{★}(1|3)$	${\psi }_n^{*}$	$[{\psi }_n^{*}\pm s_n^{*}{\xi }_{1-0.05/2}/\sqrt{n}]$
n1	0.589	0.764	0.766	1.252	[0.722;1.783]
n2	0.624	0.775	0.707	1.388	[0.974;1.802]
n3	0.679	0.767	0.795	1.361	[1.037;1.685]
n4	0.677	0.757	0.813	1.341	[1.068;1.615]
n5	0.670	0.760	0.806	1.250	[1.012;1.488]
n6	0.677	0.788	0.835	1.288	[1.126;1.451]
n7	0.694	0.793	0.834	1.273	[1.157;1.389]

For instance, we can follow the convergence of the TMLE ${\psi }_n^{*}$ toward the true risk difference 𝜓₀ in the top plot of Fig. 7.5 and in the fifth column of Table 4. Similarly, the middle plot of Fig. 7.5 and the second to fourth columns of Table 4 illustrate the convergence of $g_n^{★}$ toward G^*(θ₀), as stated in Proposition 3. What these plots and columns also teach us is that, in spite of the misspecified working model, the learned design G^*(θ₀) seems very close to the optimal treatment mechanism g^*(Q₀) for the chosen simulation scheme and working model used in our simulation study. Moreover, the last column of Table 4 illustrates how the confidence intervals $[{\psi }_n^{*}\pm s_n^{★}{\xi }_{1-0.05/2}/\sqrt{n}]$ shrink around the true risk difference 𝜓₀ as the sample size increases.

Yet, the bottom plot of Fig. 7.5 may be the most interesting of the three. It obviously illustrates the convergence of $s_n^{★2}$ toward ${\text{Var}}_{Q_0,G^{★}(\theta 0)}{D}^{★}(P_{Q_{{\theta }_0,{\epsilon }_0},G^{★}({\theta }_0)})(O)$ i·e., toward the variance under the fixed design $P_{Q_0,G^{★}({\theta }_0)}$ of the efficient influence curve at $P_{Q_{{\theta }_0,{\epsilon }_0}\cdot G^{★}({\theta }_0)}$. Hence, it also teaches us that the latter limit seems very close to the optimal asymptotic variance v^*(Q₀) for the chosen simulation scheme and working model used in our simulation study. More importantly, $s_n^{★2}$ strikingly converges to v^*(Q₀) from below. This finite sample characteristic may reflect the fact that the true finite sample variance of $\sqrt{n}({\psi }_n^{*}-{\psi }_0)$ might be lower than v^*(Q₀). Studying further this issue in depth is certainly very delicate, and goes beyond the scope of this article.

8. Simulation study of the performances of the TMLE group sequential testing procedure

In this section, we resume the simulation study undertaken in Section 7 in order to investigate the performances of the TMLE group sequential testing procedure presented in Section 6. We describe the simulation scheme in Section 8.1, then evaluate how the testing procedure behaves in terms of empirical type I and type II errors in Section 8.2 and how it behaves in terms of empirical distribution of sample size at decision in Section 8.3.

8.1. The simulation scheme, continued

We wish to test the null “Ψ(P) = ψ₀” against its alternative “Ψ(P) > ψ₀” $\left({\psi }_0=\frac{91}{72}\right)$ with prescribed type I error α = 5% and type II error β = 10% at the alternative ψ₁ = ψ₀ +0.4. Depending on whether we want to investigate the empirical behaviors of the TMLE group sequential testing procedure with respect to (i) type I or (ii) type II errors, we test M = 1000 times the above hypotheses based on 3 × M independent datasets obtained under

• (i)
  empirical type I error study:

  • i.i.d (Q₀, g^b)-balanced sampling,
  • i.i.d (Q₀, g*(Q₀))-optimal sampling,
  • $(Q_0,g_n^{★})$-adaptive sampling;
• (ii)
  empirical type II error study:

  • i.i.d (Q₁, g^b)-balanced sampling,
  • i.i.d (Q₁, g^*(Q₀))-optimal sampling,
  • $(Q_1,g_n^{★})$-adaptive sampling,

where Q₁ is defined as Q₀ in Section 7.1 with ρ = ψ₁/ψ₀ (so that Ψ(Q₁) = ψ₁).

We decide to proceed in K = 4 steps, at proportions (p₁,p₂,p3,p₄) = (0.25, 0.50,0.75,1). We choose the α-and β-spending strategies characterized by the equalities ${\displaystyle {\sum }_{l=1}^k{\alpha }_l=p_k^2\alpha }$ and ${\displaystyle {\sum }_{l=1}^k{\beta }_l=p_k^2\beta },k=1,\dots ,K$. This set of conditions characterizes the whole group sequential testing procedure, see Table 5 for the resulting numerical values.

Table 5:

Specifics of the TMLE group sequential testing procedure

Ψ0	Ψ1	Imax
1.264	1.664	57.927

Step k	1	2	3	4
Rejection boundary	2.734	2.305	2.005	1.715
α-spending	0.3125%	0.9375%	1.5625%	2.1875%

Step k	1	2	3	4
Futility boundary	−0.976	0.132	0.961	1.715
β-spending	0.625%	1.875%	3.125%	4.375%

8.2. Empirical type I and type II errors of the TMLE group sequential testing procedure

We report in Table 6 the empirical type I and type II errors obtained during the course of the simulation study. The values are strikingly close to each other in both cases. Even better, the empirical type I and type II errors obtained under $(Q,g_n^{★})$-adaptive sampling are the lowest in both cases.

Table 6:

Checking the adequateness of the type I and type II error controls on our simulated TMLE group sequential testing procedures. We test if the Binomial random numbers of times the null is falsely rejected have parameter (M, α), the alternative stating that they have parameter (M, a) with a > α. We also test if the Binomial random numbers of times the null is falsely not rejected have parameter (M, 12%), the alternative stating that they have parameter (M, b) with b > 12%. We report the empirical type I and type II errors and corresponding p-values

Sampling scheme	Type I error (Q = Q0) Empirical error (p-value)	Type II error (Q = Q1) Empirical error (p-value)
i.i.d (Q, gb)-balanced	0.040 (0.919)	0.132 (0.113)
i.i.d (Q, g★(Q))-optimal	0.043 (0.827)	0.128 (0.203)
adaptive $(Q,g_n^{★})$	0.040 (0.919)	0.126 (0.261)

Since the number of times the null was falsely rejected for its alternative is Binomial with parameter (M, α), it is possible to test rigorously if a = α (as it should) against a > α. This yields three p-values that we report in Table 6. Because there are less than 50 wrong decisions for all sampling schemes, we naturally get large p-values, confirming (if necessary) that the type I error is under control.

Similarly, the number of times the null was falsely not rejected for its alternative is also Binomial with parameter (M, b). Thus, it is possible to test rigorously if b = β (as it should) against b > β. This yields three small p-values (p < 0.001 for the i.i.d balanced sampling scheme, 0.002 for the i.i.d optimal sampling scheme and 0.003 for the adaptive sampling scheme). This confirms the impression that the number of wrong decisions are all significantly larger than what random deviations from the reference distribution are likely to allow. Put bluntly, the type II error is not under control. However, there is no real need to worry. Indeed, if one rather tests b = 12% against b > 12%, then the three p-values (reported in Table 6) are large.

In summary, for the considered scenario, the TMLE group sequential testing procedure described in Section 6.1 performs at least as well under the $(Q,g_n^{★})$-adaptive sampling scheme as under both i.i.d (Q, g^b)-balanced and (Q, g*(Q))-optimal sampling schemes. The type I error control meets the requirements. The type II error control is defective, but only slightly defective.

8.3. Empirical distribution of sample size at decision of the TMLE group sequential testing procedure

Now that we know that the TMLE group sequential testing procedure performs at least as well under adaptive sampling scheme as under both i.i.d sampling schemes, let us compare the performances in terms of sample size at decision.

For this purpose, we simply report the average sample sizes at decision for each sampling scheme when evaluating the type I and type II errors control, see Table 7. The gain in average sample size at decision obtained by resorting to the $(Q,g_n^{★})$-adaptive sampling scheme instead of the i.d.d (Q, g^b)-balanced sampling scheme is dramatic: In average, one needs approximately 16% less observations in order to reach a conclusion under the $(Q,g_n^{★})$-adaptive sampling scheme relative to the i.d.d (Q, g^b)-balanced sampling scheme. Furthermore, it appears that it is even more efficient to resort to the i.d.d (Q, g*(Q))-optimal sampling scheme: In average, one needs approximately 6% more observations in order to reach a conclusion under the i.i.d $(Q,g_n^{★})$-adaptive sampling scheme relative to the i.d.d (Q, g^b)-balanced sampling scheme.

Table 7:

Comparing the average sample sizes at decision for each sampling scheme when evaluating the type I and type II errors control.

Sampling scheme	Type I error (Q = Q0) Average sample size	Type II error (Q = Q1) Average sample size
i.i.d (Q, gb)-balanced	786.63	895.64
i.i.d (Q, g★(Q))-optimal	620.71	713.47
adaptive $(Q,g_n^{★})$	661.14	746.72

In summary, for the considered scenario, the TMLE group sequential testing procedure described in Section 6.1 reaches a decision more quickly under the $(Q,g_n^{★})$-adaptive sampling scheme than under the i.i.d (Q, g^b)-balanced sampling scheme, with a gain in average sample size at decision of approximately 16%. Furthermore, the TMLE group sequential testing procedure reaches a decision more slowly under the $(Q,g_n^{★})$-adaptive sampling scheme than under the i.i.d (Q, g*(Q))-optimal sampling scheme, but the loss in average sample size at decision reduces to approximately 6%.

A. Proofs

Let us introduce for convenience the notations θ_ε = (θ, ε), ${\theta }_0^{\epsilon }=({\theta }_0,{\epsilon }_0)$ and ${\theta }_n^{\epsilon }=({\theta }_n,{\epsilon }_n)$. Moreover, let us recall that if μ is a probability measure and f a measurable function, then μf denotes the integral ∫ fdμ.

A.1. Proof of Propositions 2 and 4, and a useful remark on Proposition 3 Proof of Proposition 2.

The cornerstone of the proof is to interpret θn as the solution in θ of the martingale estimating equation ${\displaystyle {\sum }_{i=1}^n{D}_1(\theta )(O_i,Z_i)=0}$, where Z_i (defined in (9)) is the finite dimensional summary measure of past observation O_n (i – 1) such that $g_i^{★}$ depends on O_n(i – 1) only through Z_i (hence the notation $g_i^{★}=g_{Z_i}$) and $D_1(\theta )(O,Z)={\dot{\mathcal{l}}}_{\theta ,0}(O)g^r(A|V)/g_Z(A|V)$.

Note first that, for all i ≤ n,

$$P_{Q_{0,g_i^{*}}}{D}_1({\theta }_0)=E_{P_{Q_{0,g^r}}}[{\dot{\mathcal{l}}}_{{\theta }_{\text{o}},0}(O_i)|O_n(i-1)]=0$$

by A1 (changing the order of differentiation and integration is permitted by the dominated convergence theorem, because $\mathcal{O}$ is bounded and Θ is compact—see PARAM). Observe then that, by definition of θ_n, we have:

$$\Vert \frac{1}{n}{\displaystyle \sum }_{i=1}^n{P}_{Q_0,g_i^{*}}{D}_1({\theta }_n)\Vert =\Vert \frac{1}{n}{\displaystyle \sum }_{i=1}^n{D}_1({\theta }_n)(O_i,Z_i)-P_{Q_0,g_i^{*}}{D}_1({\theta }_0)\Vert \le \theta \underset{\theta \in \Theta }{\text{sup}}\Vert \frac{1}{n}{\displaystyle \sum }_{i=1}^n{D}_1({\theta }_n)(O_i,Z_i)-P_{Q_0,g_i^{*}}{D}_1(\theta )\Vert \equiv M_{n\cdot }$$

Now, since (α) sup_θ∈Θ ∥D1(θ)∥_∞ < ∞ (because $\mathcal{O}$ is bounded and Θ is compact) and (b) the standard entropy of $\mathcal{F}=\{D_1(\theta ):\theta \in \Theta \}$ for the supremum norm satisfies $H(\mathcal{F},{\Vert \cdot \Vert }_{\infty },\epsilon )=\mathcal{O}(\text{log\hspace{0.17em}}1/\epsilon )$ (see (van der Vaart, 1998), example 19.7), we can apply (componentwise) the Kolmogorov strong law of large numbers for martingales as in Theorem 8 of (Chambaz and van der Laan, 2009). This yields the almost sure convergence of M_n, hence of n⁻¹ ${\displaystyle {\sum }_{i=1}^n{P}_{Q_0,g_i^{★}}}{D}_1({\theta }_n)$, to 0.

By a Taylor expansion of $\theta \mapsto P_{Q_0,g^r}{\dot{\mathcal{l}}}_{\theta ,0}$ at θ₀ (changing the order of differentiation and integration is permitted here too for the same reasons as above), it holds that, for all i ≤ n (recall that $P_{Q_0,g_i^{★}}{D}_1({\theta }_0)=0.$),

$$P_{Q_0,g_i^{*}}{D}_1(\theta )=(P_{Q_0,g^r}{\ddot{\mathcal{l}}}_{{\theta }_0,0})({\theta }_n-{\theta }_0)+o_P(\Vert {\theta }_n-{\theta }_0\Vert )$$

From this we deduce that $(P_{Q_0,g^r}{\ddot{\mathcal{l}}}_{{\theta }_0,0})({\theta }_n-{\theta }_0)$. converges in probability to 0. Because $P_{Q_0,g^r}{\ddot{\mathcal{l}}}_{{\theta }_0,0}$ is positive definite by A2, this implies the result.

Proof of Proposition 4.

The proof of (i) is very simple and typical of robust statistical studies. Indeed,

$$0=P_{Q_0,G*({\theta }_0)}\frac{\partial }{\partial \epsilon }{\mathcal{l}}_{{\theta }_0,\epsilon }|\epsilon ={\epsilon }_0,$$

the latter expression also writing as

$$E_{Q_0,G*({\theta }_0)}\left\{\frac{2A-1}{G*({\theta }_0)(A|V)}(Y-\overline{Q}({\theta }_0^{\epsilon })(A,W))\right\}=E_{Q_0,G*({\theta }_0)}\left\{({\overline{Q}}_0(1,W)-\overline{Q}({\theta }_0^{\epsilon })(1,W))-({\overline{Q}}_0(0,W)-\overline{Q}({\theta }_0^{\epsilon })(0,W))\right\}=\psi (Q_0)-\psi (Q_{{\theta }_0^{\epsilon }}),$$

hence the result.

The proof of (ii) fundamentally relies on the fact that ${\theta }_n^{\epsilon }$ solves the martingale estimating equation ${\displaystyle {\sum }_{i=1}^{n}D({\theta }^{\epsilon })(O_i,Z_i)=0}$, where D(θ^ε) is the extension of D₁(θ) defined in (19). Here too, $P_{Q_0,g_i^{★}}D({\theta }_0^{\epsilon })=0$ for all i ≤ n, and applying the Kolmogorov strong law of large numbers for martingales (see Theorem 8 in (Chambaz and van der Laan, 2009)) yields that $n^{-1}{\displaystyle {\sum }_{i=1}^n{P}_{Q_0,g_i^{★}}}D({\theta }_n^{\epsilon })$ converges to zero almost surely. This entails the convergence in probability of ${\theta }_n^{\epsilon }$ to ${\theta }_0^{\epsilon }$ by a Taylor expansion of $\theta \epsilon \mapsto (P_{Q_0,g^r}{\dot{\mathcal{l}}}_{\theta ,0}^{{\rm T}},P_{Q_0,G^{★}(\theta )}\frac{\partial }{\partial \epsilon }{\mathcal{l}}_{\theta ,\epsilon })$ at ${\theta }_0^{\epsilon }$. Note that A3 is a clear counterpart to A1 from Proposition 2 but that there is no counterpart to A2 from Proposition 2 in Proposition 4. Indeed, it automatically holds in the framework of the proposition that $-P_{Q_0,G^{★}({\theta }_0)}\frac{{\partial }^2}{\partial {\epsilon }^2}{\mathcal{l}}_{{\theta }_0,\epsilon }{|}_{\epsilon ={\epsilon }_0}>0$, the proof requiring that the latter quantity be different from zero.

A useful remark on Proposition 3.

We deduce in Proposition 3 the convergence in probability and in L¹ of the adaptive design $g_n^{★}$ to the limit design G^*(θ₀) from the convergence in probability of θ_n to θ₀ as obtained in Proposition 2. It is crucial for us that $n^{-1}{\displaystyle {\sum }_{i=1}^n{g}_i^{★}}$ and $n^{-1}{\displaystyle {\sum }_{i=1}^n{(g_i^{★})}^{-1}}$ also converge to G^*(θ₀) and 1/G^*(θ₀) (with an obvious notational shortcut) in probability and in L¹. Fortunately, this is true because (a) the positive random variables $g_i^{★}$ are uniformly bounded away from 0 and (b) if a sequence X_n converges in L¹ to 0 then $n^{-1}{\displaystyle {\sum }_{i=1}^n{X}_i}$ also converges in L¹ to 0 (by convexity of the L¹-norm). Let us put this in a lemma:

Lemma 1. For all $(a,v)\in \mathcal{A}\times \mathcal{V},n^{-1}{\displaystyle {\sum }_{i=1}^n{g}_i^{*}(a|v)}$ and $n^{-1}{\displaystyle {\sum }_{i=1}^n{(g_i^{*}(a|v))}^{-1}}$ converge in probability and in L¹ to G^*(θo)(a|v) and G^*(θo)(a|v)^-1.

A.2. Proof of Proposition 5

The proof of Proposition 5 still relies on the facts that (a) ${\theta }_n^{\epsilon }$ solves the martingale estimating equation ${\sum }_{i=1}^{n}D({\theta }^{\epsilon })(O_{i,}{Z}_i)=0$ and (b) $P_{Q_{0,g_i^{*}}}D({\theta }_0^{\epsilon })=0$ for all i ≤ n.

Consider the following equality:

$$\text{LHT}\equiv \frac{1}{n}{\displaystyle \sum }_{i=1}^n[D({\theta }_n^{\epsilon })(O_i,Z_i)-D({\theta }_0^{\epsilon })(O_i,Z_i)]=-\frac{1}{n}{\displaystyle \sum }_{i=1}^n[D({\theta }_n^{\epsilon })(O_i,Z_i)-P_{Q_0,g_i^{*}}D({\theta }_0^{\epsilon })].$$ (33)

A Taylor expansion of LHT at ${\theta }_0^{\epsilon }$ first yields that

$$\text{LHT}=\frac{1}{n}{\displaystyle \sum }_{i=1}^n\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon })(O_i,Z_i){|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })+o_P(\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert )=A_n({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })+B_n({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })+o_P(\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert )$$

where $A_n=E[n^{-1}{\sum }_{i=1}^n\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon })(O_{i,}{Z}_i{|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}]$ and $B_n=n^{-1}{\sum }_{i=1}^n\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon })(O_{i,}{Z}_i{|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}-A_n.$.

The matrix A_n satisfies $A_n=n^{-1}E[{\sum }_{i=1}^n{P}_{Q_{0,g_i^{*}}}\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon }){|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}]$. Now, for each i ≤ n,

$$P_{Q_0,g_i^{*}}\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon }){|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}=S_0$$ (34)

where the latter matrix, defined in (21) and independent of i, is deterministic (this is due to the weighting). Thus, A_n = S₀. Moreover, A_n = S₀ is invertible because its determinant equals the product of $P_{Q_0,{\mathit{G}}_{}^{★}}({\theta }_0)\frac{{\partial }^2}{\partial {\epsilon }^2}{\mathcal{l}}_{{\theta }_{\text{0,}\epsilon \text{|}\epsilon \text{=}{\epsilon }_0}}<0$ and

$$\text{det}{E}_{Q_0,G*({\theta }_0)}\left[{\ddot{\mathcal{l}}}_{{\theta }_0,0}(O)\frac{g^r(A|V)}{G^{*}({\theta }_0)(A|V)}\right]=\text{det}{P}_{Q_0,g^r}{\ddot{\mathcal{l}}}_{{\theta }_0,0},$$

which is negative by A2. Furthermore, (34) and A_n = S₀ (deterministic) also imply that B_n can be rewritten as

$$B_n\frac{1}{n}{\displaystyle \sum _{i=1}^n\left[\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon })(O_i,Z_i){|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}-P_{Q_0,g_i^{*}}\frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon }){|}_{{\theta }^{\epsilon }={\theta }_0^{\epsilon }}\right]}$$

and applying (componentwise) a standard law of large numbers for martingales (see for instance Theorem 2.4.2 in (Sen and Singer, 1993)) yields that B_n converges to 0 almost surely (note that ${\text{sup}}_{{\theta }^{\epsilon }\in \Theta \times \epsilon }{\Vert \frac{\partial }{\partial {\theta }^{\epsilon }}D({\theta }^{\epsilon })\Vert }_{\infty }<\infty$). We emphasize that this proves the convergence in probability of S_n to S₀ as stated in the proposition. Consequently, $\text{LHT=}{S}_0({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })+op(\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert )$ and (33) entails

$$\sqrt{n}({\theta }_n^{\epsilon }-{\theta }_n^{\epsilon })=-S_0^{-1}{M}_n+op(\sqrt{n}\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert )$$ (35)

with $M_n=n^{-1/2}{\sum }_{i=1}^n\left[D({\theta }_0^{\epsilon })(O_{i,}{Z}_i)-P_{Q_{{0g}_i^{*}}}D({\theta }_0^{\epsilon })\right]$. It mainly remains to show that M_n satisfies a central limit theorem.

For this purpose, we apply Theorem 10 in (Chambaz and van der Laan, 2009). Define $C_n={\sum }_{i=1}^n{P}_{Q_{0,g_i^{*}}}D({\theta }_0^{\epsilon })D{({\theta }_0^{\epsilon })}^{\text{Τ}}$. It holds that

$$\frac{1}{n}{C}_n=\frac{1}{n}{\displaystyle \sum _{i=1}^n{E}_{Q_0,G*({\theta }_0)}\left[\frac{\tilde{D}({\theta }_0^{\epsilon })\tilde{D}{({\theta }_0^{\epsilon })}^{\text{Τ}}(O)}{g_i^{*}(A|V)G*({\theta }_0)(A|V)}|{\mathbf{\text{O}}}_n(i-1)\right]={\displaystyle \sum _{(a,v)\in A\times \nu }{E}_{Q_0,G*({\theta }_0)}\left[\frac{\tilde{D}({\theta }_0^{\epsilon })\tilde{D}{({\theta }_0^{\epsilon })}^{\text{Τ}}(O)}{G*({\theta }_0)(A|V)}\mathbf{1}\{(A,V)=(a,v)\}\right]\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{1}{g_i^{*}(a|v)}.}}}$$ (36)

Now, by Lemma 1, (36) implies that $n^{-1}E{C}_n={\sum }_0(1+o(1))$, where Σ₀ (defined in (22)) is positive definite when A4 is met. Indeed, assume on the contrarthat there exists a vector u ≠ 0 such that $u^{\text{Τ}}{\sum }_{0}u=0$. Then necessarily $\tilde{D}{({\theta }_0^{\epsilon })}^{\text{Τ}}u=0P_{Q_0,G^{*}({\theta }_0)}$-almost surely, which contradicts A4. Using (36) again we also see that

$$\frac{1}{n}(C_n-E{C}_n)={\displaystyle \sum _{(a,v)\in A\times V}{E}_{Q_0,G*({\theta }_0)}\left[\frac{\tilde{D}({\theta }_0^{\epsilon })\tilde{D}{({\theta }_0^{\epsilon })}^{\text{Τ}}(O)}{G*({\theta }_0)(A|V)}\mathbf{1}\{(A,V)=(a,v)\}\right]\times \left(\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{1}{g_i^{*}(a|v)}-E\left[\frac{1}{n}\right]{\displaystyle \sum _{i=1}^n\frac{1}{g_i^{*}(a|v)}}}\right)}$$

from which we deduce that n⁻¹(C_n – EC_n) converges to 0 in probability by Lemma 1. Consequently, M_n converges in distribution to the centered Gaussian law with covariance matrix Σ₀. In addition, $n^{-1}{\sum }_{i=1}^{n}D({\theta }_0^{\epsilon })D{({\theta }_0^{\epsilon })}^{\text{Τ}}(O_{i,}{Z}_i)$ converges in probability to Σ₀. Since (a) ${\theta }^{\epsilon }\mapsto D({\theta }^{\epsilon })D{({\theta }^{\epsilon })}^{\text{Τ}}$ is differentiable at ${\theta }_0^{\epsilon }$, (b) its derivative is bounded, and (c) we already know that $\Vert {\theta }_0^{\epsilon }-{\theta }_0^{\epsilon }\Vert =op(1)$, yet another Taylor expansion allows to derive that Σ_n (defined in (23)) equals Σ₀ (1 + op(1)).

Let us go back to (35), knowing now that M_n satisfies a central limit theorem. We derive from it that $\sqrt{n}\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert =O_P(1)+op(\sqrt{n}\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert )$ hence $\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert =O_P(1)$. Thus, (35) does entail (20) since $P_{Q_{{\text{0}g}_i^{*}}}D({\theta }_0^{\epsilon })=0$ for all i ≤ n. The stated central limit theorem on $\sqrt{n}({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })$ readily follows.

A.3. Proof of Propositions 6 and 7

Proof of Proposition 6.

The proof of Proposition 6 is twofold and relies on the decomposition

$${\psi }_n^{*}-{\psi }_0=({\psi }_n^{*}-\text{ψ(}{Q}_n^{~}\text{)})+(\text{ψ(}{Q}_n^{~}\text{)-}{\psi }_0),$$ (37)

where

$$Q_n^{~}=(Qw(P_0),Q_{Y|A,W}({\theta }_n^{\epsilon })).$$ (38)

First, a continuity argument and the convergence in probability of the stacked estimator ${\theta }_n^{\epsilon }$ to ${\theta }_0^{\epsilon }$ entail the convergence in probability of $\text{ψ(}{Q}_n^{~}\text{)}$ to $\text{ψ(}{Q}_{{\theta }_0^{\epsilon }}\text{)=}{\psi }_0$ (the equality holds by (i) in Proposition 4). The conclusion follows because ${\psi }_n^{*}\text{-ψ(}{Q}_n^{~})$ converges almost surely to zero. Indeed,

$${\psi }_n^{*}-\text{ψ(}{Q}_n^{~}\text{)=(}{P}_{W,n}\text{-}{P}_{W,0}\text{)}P{\theta }_n^{\epsilon }$$ (39)

where, for all ${\theta }^{\epsilon }\in \Theta \times \epsilon ,q{\theta }^{\epsilon }=\overline{Q}({\theta }^{\epsilon })(1,\cdot )-\overline{Q}({\theta }^{\epsilon })(0,\cdot )$ and, with a slight abuse of notation, P_W,_n (respectively, P_W,0) denotes the empirical (respectively, true) marginal distribution of W. Thus,

$$\left|{\psi }_n^{*}-\text{ψ(}{Q}_n^{~}\text{)}\right|\le \underset{{\theta }^{\epsilon }\in \Theta \times \epsilon }{\text{sup}}\left|(P_{W,n}-P_{W,0})q{\theta }^{\epsilon }\right|,$$ (40)

where (a) ${\text{sup}}_{{\theta }^{\epsilon }\in \Theta \times \epsilon }{\Vert q{\theta }^{\epsilon }\Vert }_{\infty }<\infty$ and (b) the standard entropy of $\mathcal{F}=\{q{\theta }^{\epsilon }:{\theta }^{\epsilon }\in \Theta \times \epsilon \}$ for the supremum norm satisfies $H(\mathcal{F},{\Vert \cdot \Vert }_{\infty },\epsilon )=\mathcal{O}(\text{log}1/\epsilon )$ (see (van der Vaart, 1998), example 19.7). Therefore, the class $\mathcal{F}$ is P_W,₀-Glivenko-Cantelli and the right-hand side term of (40) converges to 0 P_W,₀-almost surely by the Glivenko-Cantelli theorem (i.i.d framework; see for instance Theorem 19.4 in (van der Vaart, 1998)).

Proof of Proposition 7.

The proof of (26) relies again on (37). It is easy to derive the asymptotic linear expansion of the first term. Indeed, we can rewrite (39) as

$$\begin{array}{lll}\sqrt{n}({\psi }_n^{*}-\text{ψ(}{Q}_n^{~}\text{)})\hfill & =\hfill & \sqrt{n}(P_{W,n}-P_{W,0})q{\theta }_0^{\epsilon }+\sqrt{n}(P_{W,n}-P_{W,0})(q{\theta }_n^{\epsilon }-q{\theta }_0^{\epsilon })\hfill \\ \hfill & =\hfill & \sqrt{n}{P}_{W,n}(q{\theta }_0^{\epsilon }-P_{W,0q{\theta }_0^{\epsilon }})+oP(1)\hfill \\ \hfill & =\hfill & \sqrt{n}{P}_{W,n}{D}_1(Q_{{\theta }_0^{\epsilon }})+op(1),\hfill \end{array}$$ (41)

which holds subject to checking that $\sqrt{n}(P_{W,n}-P_{W,0})(q{\theta }_n^{\epsilon }-q{\theta }_0^{\epsilon })=op(1)$. Since the class $\mathcal{F}$ introduced previously for the proof of Proposition 6 is also P_W,₀-Donsker, this is a consequence of Lemma 19.24 in (van der Vaart, 1998), provided that $X_n=P_{W,0}{(q{\theta }_n^{\epsilon }-q{\theta }_0^{\epsilon })}^2$ converges in probability to 0. Now, ${\theta }_n^{\epsilon }$ converges to ${\theta }_0^{\epsilon }$ in probability, and the function $(w,{\theta }^{\epsilon })\mapsto q{\theta }^{\epsilon }(w)$ continuously maps $\mathcal{W}\times \Theta \times \epsilon$ onto $ℝ$ (thus it is uniformly bounded). Consequently, X_n (which is obviously non-negative) is upper bounded, so that it is equivalent to show that X_n converges to 0 in L¹. By the Fubini theorem,

$$E{X}_n={\displaystyle \int E[{(q{\theta }_n^{\epsilon }(w)-q{\theta }_n^{\epsilon }(w))}^2]d{P}_{W,0}(w).}$$

By the continuous mapping theorem (see Theorem 1.3.6 in (van der Vaart and Wellner, 1996)), ${\left(q{\theta }_n^{\epsilon }(w)-q{\theta }_0^{\epsilon }(w)\right)}^2$ converges to 0 in probability (hence in L¹, the variables being bounded) for all $w\in \mathcal{W}$. Therefore, the integrand of the right-hand side integral in the previous display converges pointwise to 0. Since it is also bounded, we conclude by the dominated convergence theorem that EX_n converges to 0. This completes the study of the first term of the decomposition (37).

Regarding the second term of the latter decomposition, we derive its asymptotic linear expansion by the delta-method (see Theorem 3.1 in (van der Vaart, 1998)) from that of $\sqrt{n}({\theta }_n^{\epsilon }-{\theta }_0^{\epsilon })$, see (20). Specifically,

$$\sqrt{n}(\text{ψ(}{Q}_n^{~}\text{)-}{\psi }_0)=\sqrt{n}(\varphi ({\theta }_n^{\epsilon })-\varphi ({\theta }_n^{\epsilon }))={\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}\frac{1}{\sqrt{n}}{\displaystyle \sum _{i=1}^{n}D({\theta }_0^{\epsilon })(O_i,Z_i)+op(1).}$$ (42)

Combining (41) and (42) yields the wished asymptotic linear expansion (26) and the closed-form expression of the related influence function IC as given in (27).

A central limit theorem for real-valued martingales (see Theorem 9 in (Chambaz and van der Laan, 2009)) applied to (26) yields the stated convergence and validates the use of $s_n^2$ as an estimator of the asymptotic variance. To see this, note that $P_{Q_{0,g_i^{*}}}\text{IC=0}$ for all i ≤ n and define ${\text{c}}_n={\sum }_{i=1}^n{P}_{Q_{{\text{0}g}_i^{*}}}{\text{IC}}^2$. Now we emphasize that, for every i ≤ n, firstly

$$P_{Q_0,g_i^{*}}{D}_1^{*}{(Q_{{\theta }_0^{\epsilon }})}^2=P_{Q_0,G*({\theta }_0)}{D}_1^{*}{(Q_{{\theta }_0^{\epsilon }})}^2,$$

secondly

$$\begin{array}{lll}2P_{Q_0,g_i^{*}}{D}_1^{*}(Q_{{\theta }_0^{\epsilon }}){\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}D({\theta }_0^{\epsilon })\hfill & =\hfill & 2{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}{P}_{Q_0,g_i^{*}}{D}_1^{*}(Q_{\theta }{}_{{}_0^{\epsilon }})D({\theta }_0^{\epsilon })\hfill \\ \hfill & =\hfill & 2{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}{P}_{Q_0,G*({\theta }_0)}\left[\frac{D_1^{*}(Q_{{\theta }_0^{\epsilon }})\tilde{D}({\theta }_0^{\epsilon })}{G*({\theta }_0)}\right]\hfill \end{array}$$

and thirdly

$$P_{Q_0,g_i^{*}}{({\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}D({\theta }_0^{\epsilon }))}^2={\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}(P_{Q_0,g_i^{*}}D({\theta }_0^{\epsilon })D{({\theta }_0^{\epsilon })}^{\text{Τ}}){({\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1})}^{\text{Τ}}.$$

By combining the last three equalities, we thus obtain that

$$\frac{1}{n}{c}_n=P_{Q_0,G*({\theta }_0)}{D}_1^{*}{(Q_{{\theta }_0^{\epsilon }})}^2+2{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}{P}_{Q_0,G*({\theta }_0)}\left[\frac{D_1^{*}(Q_{{\theta }_0^{\epsilon }})\tilde{D}({\theta }_0^{\epsilon })}{G*({\theta }_0)}\right]+{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}\frac{1}{n}{C}_n{({\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1})}^{\text{Τ}}$$ (43)

where Cn was introduced in the proof of Proposition 5. Since we already showed in the latter proof that n⁻¹EC_n = Σ₀(1 + o_P(1)), (43) notably yields the convergence of n⁻¹Ec_n toward

$$P_{Q_0,G*({\theta }_0)}{D}_1^{*}{(Q_{{\theta }_0^{\epsilon }})}^2+2{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}{P}_{Q_0,G*({\theta }_0)}\left[\frac{D_1^{*}(Q_{{\theta }_0^{\epsilon }})\tilde{D}({\theta }_0^{\epsilon })}{G*({\theta }_0)}\right]+{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}{\Sigma }_0{({\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1})}^{\text{Τ}}=P_{Q_0,G*({\theta }_0)}{\left(D_1^{*}(Q_{{\theta }_0^{\epsilon }})+{\varphi }_{{\theta }_0^{\epsilon }}^{\text{'}\text{Τ}}{S}_0^{-1}\frac{\tilde{D}({\theta }_0^{\epsilon })}{G*({\theta }_0)}\right)}^2\equiv s^2$$

which is positive when A4 is met. In addition, (43) also entails that n⁻¹(c_n-Ec_n) converges to 0 in probability, therefore implying the convergence in distribution of $\sqrt{n}({\psi }_n^{*}-{\psi }_0)$ to the centered Gaussian distribution with variance s² as well as the convergence in probability of $n^{-1}{\displaystyle {\sum }_{i=1}^n\text{IC\hspace{0.17em}}{(O_i,Z_i)}^2}$ to s². Since (a) ${\theta }^{\epsilon }\mapsto D_1^{★}(Q_{{\theta }^{\epsilon }})$ and θ^ε ↦ D(θ^ε) are differentiable at ${\theta }_0^{\epsilon }$,(b) both with uniformly bounded derivatives, (c) we already know that $\Vert {\theta }_n^{\epsilon }-{\theta }_0^{\epsilon }\Vert =oP(1)$, and (d) ${\varphi }_{{\theta }_n^{\epsilon }}^{\prime }$ and $S_0^{-1}$ are consistently estimated by ${\varphi }_n^{\prime }$ and $S_n^{-1}$, yet another Taylor expansion (and the continuous mapping theorem, see for instance Theorem 1.3.6 in (van der Vaart and Wellner, 1996)) finally yields the stated convergence in probability of $s_n^2$ to s², therefore completing the proof.

A.4. Proof of Proposition 8

The fact that $\text{Ψ}(Q_{f/\sqrt{n}})>\Psi (Q_0)$ for n large enough is a consequence of the expansion $\text{Ψ}(Q_{f/\sqrt{n}})=\text{\hspace{0.17em}Ψ\hspace{0.17em}}(Q_0)+P_{Q_0,G^{★}({\theta }_0)}{D}^{★}(P_{Q_0,G^{★}({\theta }_0)})f/\sqrt{n}+o(1/\sqrt{n})$, which holds because Ψ is pathwise differentiable at P_Q0,g (for any g) relative to the maximal tangent space, with efficient influence curve D*(Q₀, g). The rest of Proposition 8 is a corollary of the following lemma.

Lemma 2. Let Λ_n be the log-likelihood ratio of the $(Q_{f/\sqrt{n}},g_n^{★})$ experiment relative to the $(Q_0,g_n^{★})$ experiment. Under $(Q_0,g_n^{★})$, the vector $(\sqrt{n_1}({\psi }_{n_1}^{★}-\text{\hspace{0.17em}Ψ}(Q_0)),\dots ,(\sqrt{n_K}({\psi }_{n_K}^{*}-\text{\hspace{0.17em}Ψ}(Q_0)),{\Lambda }_n)$ converges in law to the Gaussian distribution with mean $(0,\dots ,0,-\frac{1}{2}{P}_{Q_0,G^{★}({\theta }_0)}{f}^2)$ and covariance matrix

$$\left(\begin{array}{cc}{P}_{Q_0,G*({\theta }_0)}{\text{IC}}^2\times C& P_{Q_0,G*({\theta }_0)}ICf\times {(\sqrt{pK})}^{\text{Τ}}\\ P_{Q_0,G*({\theta }_0)}ICf\times (\sqrt{p1},\mathrm{...},\sqrt{pK})& P_{Q_0,G*({\theta }_0)}{f}^2\end{array}\right).$$

In particular, the $(Q_{f/\sqrt{n}},g_n^{★})$ and $(Q_0,g_n^{★})$ experiments are mutually contiguous.

The limiting distribution of (T₁,…,T_K) under $(Q_0,g_n^{★})$ is easily derived from Lemma 2. Le Cam’s third lemma (see Example 6.7 in (van der Vaart, 1998)) solves the problem of obtaining the limiting distribution of (T₁,…,T_K) under $(Q_{f/\sqrt{n}},g_n^{★})$ from that under $(Q_0,g_n^{★})$. Only the asymptotic means are different. We do not repeat the proof here, as it is exactly the same (up to some minor variations in notation) as the proof of Lemma 5 in (Chambaz and van der Laan, 2011).

Proof of Lemma 2.

Since f is bounded and $E_{P_{Q_0},G^{★}({\theta }_0)}[f(O)|A,W]=0$ (here, one could equivalently use the notation E_Q0), we can assume without loss of generality that the fluctuation $\{Q_0(h):h\in \mathcal{H}\}$ is characterized by

$$\frac{d{Q}_0(h)}{d{Q}_0}=(1+hf(O))$$

for all $h\in \mathcal{H}=(-1/{\Vert f\Vert }_{\infty },1/{\Vert f\Vert }_{\infty })$. Let us denote by q_n and q₀ the conditional densities of Y given (A, W) under $Q_{f/\sqrt{n}}$ and Q₀.

The log-likelihood ratio ${\Lambda }_n={\displaystyle {\sum }_{i=1}^n\text{log\hspace{0.17em}}{q}_n/q_0(O_i})$, by conditional independence of O₁,…, O_n given ((A₁, W₁),…, (A_n,W_n)) and because the marginal distribution of W is the same under $Q_{f/\sqrt{n}}$ as under Q₀, (see (4) and note that the $g_i^{★}(A_i|W_i)$ factors cancel out). The fluctuation is chosen in such a way that $q_n/q_0=(1+f/\sqrt{n})$, hence ${\Lambda }_n={\sqrt{n}}^{-1}{\displaystyle {\sum }_{i=1}^{n}f(O_i)-\frac{1}{2}{n}^{-1}{\displaystyle {\sum }_{i=1}^n{f}^2}(O_i)+n^{-1}}{\displaystyle {\sum }_{i=1}^n{f}^2}(O_i)R(f(O_i)/\sqrt{n})$, where the function R is characterized by $\text{log}(1+x)=x-\frac{1}{2}{x}^2+x^{2}R(x)(\text{all\hspace{0.17em}}x>-1)$. In particular, R is increasing and R(x) → R(0) = 0 when x → 0. Since f is bounded, the expansion is valid for n large enough. Moreover, the last term satisfies

$$\left|\frac{1}{n}{\displaystyle \sum _{i=1}^n{f}^2(O_i)R(f(O_i)/\sqrt{n})}\right|\le \frac{1}{n}{\displaystyle \sum _{i=1}^n{f}^2(O_i)\times R({\Vert f\Vert }_{\infty }/\sqrt{n}),}$$

so that if $n^{-1}{\displaystyle {\sum }_{i=1}^n{f}^2}(O_i)=O_P(1)$ then it is o_P(1). Furthermore, the Kolmogorov law of large numbers for martingales implies that $n^{-1}{\displaystyle {\sum }_{i=1}^n{f}^2}(O_i)=n^{-1}{\displaystyle {\sum }_{i=1}^n{f}^2}{P}_{Q_0,g_i^{★}}{f}^2+oP(1)=P_{Q_0,{\overline{g}}_n^{★}}{f}^2+oP(1)$, where ${\overline{g}}_n^{★}=n^{-1}{\displaystyle {\sum }_{i=1}^n{g}_i^{★}}$ Denote Var(f)(A, W) = EQ₀ [f² (O)|A, W]. We have that

$$\begin{array}{l}\begin{array}{lll}{P}_{Q_{0,{\overline{g}}_n^{*}}}{f}^2\hfill & \equiv \hfill & E_{Q_{0,}{\overline{g}}_n^{*}}[f^2(O_n)|{\mathbf{\text{O}}}_n(n-1)]=E_{Q_{0,}{\overline{g}}_n^{*}}[Var(f)(A,W)(O_n)|{\mathbf{\text{O}}}_n(n-1)]\hfill \\ \hfill & =\hfill & E_{Q_0}[{\overline{g}}_n^{*}(1|V)Var(f)(1,W)+{\overline{g}}_n^{*}(0|W)Var(f)(1,V)|{\mathbf{\text{O}}}_n(n-1)]\hfill \end{array}\\ \end{array}$$

for V the relevant subvector of W upon which each $g_i^{*}$ depends. By Lemma 1, ${\overline{g}}_i^{*}$ converges in probability to G*(θ₀). Consequently, the continuous mapping theorem (see Theorem 1.3.6 in (van der Vaart and Wellner, 1996)) yields that $P_{Q_{0,{\overline{g}}_n^{*}}}{f}^2=P_{Q_0,G*({\theta }_{\text{o}})}{f}^2+o_P(1)=O_P(1)$. In summary, we obtain the following asymptotic linear expansion:

$${\text{Λ}}_n=\frac{1}{\sqrt{n}}{\displaystyle \sum _{i=1}^{n}f(O_i)-\frac{1}{2}}{P}_{Q_0,G*({\theta }_0)}{f}^2+op(1).$$ (44)

Introduce now $Z_i^{\prime }=(1\{i\le n_1\},\mathrm{...},1\{i\le n_K\})$ and the bounded (and measurable) function F such that $F(O_i,Z_i,Z_i^{\prime })=(Z_i^{\prime }\text{IC(}{O}_i,Z_i\text{),}f(O_i))$. We show that $M_n=n^{-1}{\sum }_{i=1}^n[F(O_i,Z_i,Z_i^{\prime })-P_{Q_{0,g_i^{*}}}F]=n^{-1}{\sum }_{i=1}^{n}F(O_i,Z_i,Z_i^{\prime })(\text{since\hspace{0.17em}}{P}_{Q_{0,g_i^{*}}}F=P_{Q_{\text{o},g_i^{*}}}f=0\text{\hspace{0.17em}for\hspace{0.17em}all\hspace{0.17em}}i\le n)$ satisfies a central limit theorem. The proof is very similar to the corresponding part of proof of Lemma 5 in (Chambaz and van der Laan, 2011), hence we only give an outline, focusing on the differences. First, it holds for every 1 ≤ k, l ≤ K that $n^{-1}{\sum }_{i=1}^{n_k}{P}_{Q_{\text{o},g_i^{*}}}\text{IC}f=p_k{P}_{Q_0,G*({\theta }_{\text{o}})}\text{IC}f+o_P(1),n^{-1}{\sum }_{i=1}^{n_k\wedge n_l}{P}_{Q_{\text{o},g_i^{*}}}{\text{IC}}^2=p_{k\wedge l}{P}_{Q_0,G*({\theta }_{\text{o}})}{\text{IC}}^2+o_P(1),\text{\hspace{0.17em}and\hspace{0.17em}}{n}^{-1}{\sum }_{i=1}^n{P}_{Q_{\text{o},g_i^{*}}}{f}^2=P_{Q_0,G*({\theta }_{\text{o}})}{f}^2+o_P(1)$. This entails that the matrix $E{n}^{-1}{\sum }_{i=1}^n{P}_{Q_{\text{o},g_i^{*}}}{F}^{\top }F$ converges to

$$\left(\begin{array}{cc}{P}_{Q_0,G*({\theta }_0)}{\text{IC}}^2\times {(pk\wedge l)}_{k,l\le K}& P_{Q_0,G*({\theta }_0)}ICf\times {(p_1,\mathrm{...},p_K)}^{\text{Τ}}\\ P_{Q_0,G*({\theta }_0)}\text{IC}f\times (p_1,\mathrm{...},p_K)& P_{Q_0,G*({\theta }_0)}{f}^2\end{array}\right),$$

which is positive definite if and only if its determinant is positive. The latter determinant equals a positive constant times $\left[P_{Q_0,G*({\theta }_{\text{o}})}{f}^2-{(P_{Q_0,G*({\theta }_{\text{o}})}\text{IC}f)}^2/P_{Q_0,G*({\theta }_{\text{o}})}\text{I}{\text{C}}^2\right]$, which is positive too by the Cauchy-Schwarz inequality (f is not proporional to $D^{★}\left(P_{Q_0,G^{★}({\theta }_0)}\right)$). Therefore, Theorem 8 in (Chambaz and van der Laan, 2011) applies and $\sqrt{n}{M}_n$ converges in law to the centered Gaussian distribution with the covariance matrix given in the above display. The conclusion readily follows.

Figure 2: Illustrating the TMLE procedure under $(Q_0,g_n^{★})$-adaptive sampling scheme.

These three plots illustrate how the TMLE procedure behaves (on a simulated trajectory) as the sample size (on x-axis, logarithmic scale; the vertical grey lines indicate the sample sizes n_i, i = 1,..., 7) increases. Top plot: we represent the sequence ${\psi }_n^{*}(O_{n_7}^1(n))$ at intermediate sample sizes n. The horizontal grey line indicates the true value of the risk difference 𝜓₀. Middle plot: we represent the three sequences $g_n^{★}(1|1)=G^{★}({\theta }_n(O_{n_7}^1(n)))(1|1)$ (bottom curve), $g_n^{★}(1|2)=G^{★}({\theta }_n(O_{n_7}^1(n)))(1|2)$ (middle curve) and $g_n^{★}(1|3)=G^{★}({\theta }_n(O_{n_7}^1(n)))(1|3)$ (top curve) at intermediate sample sizes n. The three horizontal grey lines indicate the optimal allocation probabilities g*(Q₀)(1|1) (bottom line), g*(Q₀)(1|2) (middle line) and g*(Q₀)(1|3) (top line). Bottom plot: we represent the sequence $s_n^{★2}$ of estimated asymptotic variance of $\sqrt{n}({\psi }_n^{*}-{\psi }_0)$ at intermediate sample sizes n. The horizontal grey line indicates the value of the optimal variance v*(Q₀). See also Table 4.