“Within‐Trial” Prognostic Score Adjustment Is Targeted Maximum Likelihood Estimation

ABSTRACT

Adjustment for “super” or “prognostic” composite covariates has become more popular in randomized trials recently. These prognostic covariates are often constructed from historical data obtained from previous clinical trials or registries by fitting a predictive model of the outcome on the raw covariates. A natural question that we have been asked by applied researchers is whether this can be done without the historical data: can the prognostic covariate be constructed or derived from the trial data itself, possibly using different folds of the data, before adjusting for it? Here we clarify that such “within‐trial” prognostic adjustment is nothing more than a form of targeted maximum likelihood estimation (TMLE), a well‐studied procedure that typically improves the power of trial analyses. We therefore argue that there is no reason to reinvent the wheel: within‐trial prognostic score adjustment should be referred to as TMLE, without qualification.

1. Introduction

Recently, there has been an increase in the exploration of “super covariates” or “prognostic covariates” [1] as effective tools for covariate adjustment in statistical analysis in randomized clinical trials (RCTs). The idea, which in principle dates back at least to Tukey [2], is to utilize historical data to develop a single prognostic covariate, facilitating an increase in efficiency with reduced risk of overfitting. This strategy effectively combines insights from earlier trials to strengthen the overall study power [1, 2, 3, 4]. Schuler et al. [4] formalized some of these ideas into the PROCOVA ¹ approach, which allows for adjustment of a prognostic score in a linear analysis and translates the potential power gains to sample size gains. We will refer to this method as standard prognostic score adjustment. This method preserves the randomization process and mitigates the increase in type I errors seen for other data fusion methods [5, 6, 7]. Schuler et al. [4] demonstrated that the estimate of the average treatment effect (ATE) derived from this method is efficient, given the assumption of homogeneous treatment effects. Højbjerre‐Frandsen et al. [8] examine the practical aspects associated with the application of this method.

Another example of how to increase efficiency by adjusting for baseline covariates is targeted maximum likelihood estimation (TMLE) [9, 10]. The method is suitable for a wide range of data types and does not require the availability of historical data. There are numerous extensions that utilize cross‐validation to enhance efficiency [11, 12, 13, 14, 15, 16, 17]. In many instances, the estimator achieves local semi‐parametric efficiency, with discussions by [9, 10, 18] outlining the requirements for TMLE to reach non‐parametric efficiency. Additionally, Liao et al. [19] expanded the method of prognostic score adjustment to encompass estimators that are already efficient, which is usually the case for TMLE under an RCT.

In this paper we explore “within‐trial” prognostic score adjustment. This method leverages the ideas of prognostic score adjustment, but eliminates the need for historical data. We start by setting the theoretical framework for ATE estimation in an RCT. We next describe how a linear model can be used for ATE estimation, following how the efficiency of this procedure can be increased using prognostic scores. We next explain TMLE for ATE estimation in a 1:1 RCT setting and use this to argue that within‐trial prognostic adjustment is a type of TMLE for ATE estimation in RCTs. Finally, we discuss the impact of the equivalence between the two methods and compare this to the standard prognostic score adjustment proposed by Schuler et al. [4].

2. Setting and Notation

Our setting is a two‐arm trial involving $n$ participants $O_i=\left(W_i,A_i,Y_i\right)$, which are drawn independently and identically distributed from some population distribution. ² We refer to a generic observation as $O=\left(W,A,Y\right)$ without index $i$. Here $Y$ represents a continuous primary endpoint variable and $W$ is a vector of $p$ baseline covariates. $A$ is the treatment indicator, which is $1$ if the participant is randomized to the new treatment and $0$ otherwise. We do not impose any parametric assumptions on the distribution of $Y$ given $\left(A,W\right)$. The trial dataset is represented as $\left(\mathbb{W},\mathbb{A},\mathbb{Y}\right)\in {\mathcal{W}}^n\times {\left\{0,1\right\}}^n\times {\mathrm{ℝ}}^n$, where $\mathcal{W}$ is the domain of $W$. The sizes of the treatment and control groups are represented as $n_1$ and $n_0$, respectively. Throughout the paper, the term “asymptotic” refers to sampling $n$ observations of $O$ as $n\to \infty$.

Each participant is associated with two potential outcomes: $Y\left(1\right)$ under the new treatment and $Y\left(0\right)$ under control [20, 21, 22]. The estimand of interest is the causal average treatment effect (ATE), defined mathematically as:

$${\mathrm{\Psi }}^{*}=\mathbb{E}\left[Y\left(1\right)-Y\left(0\right)\right].$$ (1)

without further assumptions, there is no way to estimate this causal effect because we can not observe both potential outcomes simultaneously, even with infinite draws from the $\left(Y,A,W\right)$ data structure. However, there are multiple sets of assumptions that one can impose on the causal data generating process $\left(Y\left(1\right),Y\left(0\right),A,W\right)$ so that $\mathbb{E}\left[Y\left(a\right)\right]$ is equal to some parameter (functional) of the observed data distribution, assuming that the observed outcome is $Y=Y\left(A\right)$ (whichever potential outcome corresponds to observed treatment).

The most parsimonious of these is assuming simple randomization. Due to randomization, the potential outcomes $Y\left(0\right)$ and $Y\left(1\right)$ are independent of the treatment allocation $A$ and the exposure probabilities $1>P\left(A=a\right)={\pi }_a>0$ (for $a\in \left\{0,1\right\}$) are known. In this case the identifying functional becomes $\mathbb{E}\left[Y\left(A\right)\right]=\mathbb{E}\left[\mathbb{E}\left[Y|A,W\right]\right]$. In much of the literature, this is called the statistical estimand (in contrast to the causal estimand). The two are identical under the identifying assumptions but the form that the statistical estimand takes depends on what the identifying assumptions are. A well‐known benefit of identification via randomization is that the randomization assumption can be enforced by design, as is done in a randomized trial. Thus the causal estimand (1) is identified by a corresponding statistical estimand:

(2)

where $\mu \left(a,W\right)=\mathbb{E}\left[Y|A=a,W\right]$ is the conditional mean function. In the remainder of the paper we treat the identifying functional (statistical estimand) $\mathrm{\Psi }$ as the target parameter and focus on estimators and inference for this under simple randomization.

3. Ate Estimation Methods in Randomized Trials

In this section we explain three approaches to estimate $\mathrm{\Psi }$ in RCTs: the standard ANCOVA (plug‐in) estimator, prognostic score adjustment using historical data, and TMLE. These definitions are used to set up the equivalence result in Section 4.

3.1. ANCOVA Estimator

A simple linear model including an intercept and a treatment term is often used to estimate the ATE for a continuous endpoint in the pharmaceutical industry. In such a model the mean of the endpoint is modeled as $\mathbb{E}\left[Y|A,W\right]=\mu \left(A,W\right)={\beta }_0+A{\beta }_A+X{\beta }_X$ where $X$ consists of some arbitrary terms constructed from the covariates $W$. A common choice is the main‐terms regression $X=W$, but $X$ may more generally contain nonlinear expansions and interactions, including with the treatment. Strictly speaking we should write $X\left(A,W\right)$ to denote this dependence but $X$ is a convenient abbreviation. Least squares is used to estimate the coefficients $\beta$ (which corresponds to maximum likelihood with assumed Gaussian errors). When there are no terms in $X$, the estimator is often known as the unadjusted estimator, analysis of variance (ANOVA), or difference‐in‐means, and if $X$ includes covariates the estimator is known as analysis of covariance (ANCOVA) or just regression adjustment.

If the design matrix includes no treatment–covariate interaction terms, or if the covariates are centered, then the ATE can be consistently estimated in an RCT as $\widehat{\mathrm{\Psi }}={\widehat{\beta }}_A$, the coefficient on the treatment indicator [16]. The robust (Huber–White) standard error may typically be used for inference if some care is taken with centering the covariates [8, 23, 24, 25, 26].

While this is the standard presentation of ANCOVA, here we give a complementary account following Rosenblum and van der Laan [16] that is algebraically equivalent but reveals the close connection between ANCOVA, G‐computation, and TMLE. In this account, the ANCOVA estimate is obtained by the following plug‐in procedure:

1. Use least squares to estimate ${\beta }_0$ and $\beta$.

2. Extract the counterfactual predictions using the estimated conditional mean functions $\widehat{\mu }\left(a,w\right)={\widehat{\beta }}_0+a{\widehat{\beta }}_A+x{\widehat{\beta }}_X$, that is,

   $${\widehat{\mathrm{\Psi }}}_a=\frac{1}{n}\sum _{i=1}^n\widehat{\mu }\left(a,W_i\right),a\in \left\{0,1\right\}.$$ (3)

3. Get the ATE estimate by plugging in $\widehat{\mathrm{\Psi }}={\widehat{\mathrm{\Psi }}}_1-{\widehat{\mathrm{\Psi }}}_0$.

4. The sampling variance of the influence function for this estimator gives a consistent estimate of the asymptotic variance,

   $${\widehat{\sigma }}_{\infty }^2=\frac{1}{n}\sum _{i=1}^n{\left({\widehat{\varphi }}_1\left(W_i,A_i,Y_i\right)-{\widehat{\varphi }}_0(W_i,A_i,Y_i)\right)}^2,$$ (4)

where ${\widehat{\varphi }}_a\left(W_i,A_i,Y_i\right)=\frac{1_a\left(A_i\right)}{{\pi }_a}\left(Y-\widehat{\mu }\left(a,W_i\right)\right)+\left(\widehat{\mu }\left(a,W_i\right)-{\widehat{\mathrm{\Psi }}}_a\right)$. Confidence intervals and p‐values may then be constructed from the approximate standard error $\widehat{\sigma }={\widehat{\sigma }}_{\infty }/\sqrt{n}$.

Viewing ANCOVA in this light makes it clear that it is a specific case of G‐computation in the context of an RCT using a linear regression model as the estimate of the conditional mean of the outcome.

3.2. Prognostic Adjustment

The plug‐in formulation of the ANCOVA estimator illustrates that good estimation of the ATE depends on good estimation of the conditional mean $\mu$. Indeed, adjustment works by explaining away variability in the outcome with variability in the covariates, leaving the effect of treatment more clearly visible.

The idea of prognostic adjustment is to improve this estimate with historical data while protecting from possible type I error inflation. Instead of directly using the external data in the trial analysis, the external data are used to learn a separate prediction model, the prognostic model, which predicts outcomes from covariates and possibly treatment. This model is used on the trial data to generate predictions (the “prognostic score”), which are then included as a covariate vector in a standard ANCOVA analysis.

Formally, to define a prognostic score we introduce the stochastic variable $D$, which is $1$ if the participant comes from the new trial and $0$ if the observation is from the historical data. The prognostic score is defined as the expected observed outcome conditional on the covariates

$$\begin{array}{l}{\rho }_D\left(W,A\right)≔\mathbb{E}\left[Y|W,A,D\right].\end{array}$$ (5)

This true expected value is called the oracle prognostic score. The trial‐specific conditional mean which we are interested in to estimate the ATE can be expressed as $\mu \left(A,W\right)={\rho }_1\left(W,A\right)$. Thus if the external ($D=0$) population is similar to the trial population, ${\rho }_0\approx {\rho }_1$ and we can leverage the external data to improve estimation of $\mu$.

Having access to historical data $\left(\tilde{\mathbb{W}},\tilde{\mathbb{A}},\tilde{\mathbb{Y}}\right)\in {\mathcal{W}}^{\tilde{n}}\times {\left\{0,1\right\}}^{\tilde{n}}\times {\mathrm{ℝ}}^{\tilde{n}}$ obtained for $\tilde{n}$ participants, estimators of the prognostic scores can be obtained by applying a machine learning algorithm to this data. Usually, there are no treatment units in the historical data meaning that $A=0$ for all participants and only one estimator ${\widehat{\rho }}_0\left(W,0\right)$ of the prognostic score would be obtained. However, if prior treatment‐arm data are available, one may also estimate (and later adjust for) the treatment‐arm prognostic score ${\rho }_0\left(W,1\right)$.

To estimate the treatment effect in the trial, the prognostic score is evaluated for all observations in the trial data and used as a covariate in a standard ANCOVA analysis [1, 2, 3, 4]. The prognostic score may either be used as the only adjustment covariate or in addition to some or all of the other covariates, possibly including covariates that were not inputs to the prognostic score (e.g., because they were not measured in the historical sample).

If the external and trial populations are similar, the prognostic score is likely to be a powerful and predictive summary measure which will improve estimation of the true outcome regression in the trial, reduce unexplained variance, and therefore increase power for the ATE analysis [1, 3, 4]. Any finite‐sample gain in efficiency depends on the prognostic model's ability to capture nonlinearities and interaction effects that are missed by the linear model or that would not be learnable from a (usually smaller) trial dataset. For exact statements on the requirements for efficiency gain see Schuler et al. [4], Schuler [27], Tsiatis [28], and Højbjerre‐Frandsen et al. [8].

When using external data, power gains often come at the expense of type I error control, but this is not the case for prognostic adjustment. From the perspective of the trial data‐generating process (fixing the external data), the prognostic model is a prespecified, fixed transformation of the predictors and therefore works like any other prespecified term in the ANCOVA design matrix. By this argument, adjusting for it cannot (asymptotically) incur type I error for the ATE when using, for example, Wald‐type inference based on the robust or influence‐function based standard error.

Whether to adjust for covariates besides the prognostic score is a matter of judgment. Asymptotically, more covariates cannot hurt efficiency and may substantially improve it if the prognostic model is not accurate or does not ingest all of the available covariates in the trial. Therefore when the number of raw covariates is small or the utility of the prognostic model is in doubt, raw covariates should be included. It is also sensible to include only some subset of covariates known a priori to be highly predictive. However, including too many raw covariates besides the prognostic score can hurt finite‐sample efficiency due to the increase in the “degrees of freedom” of the model. In cases where the prognostic score is thought to be a good summary and the number of raw covariates is very high, it may be sensible to adjust only for the prognostic score in the ANCOVA analysis.

3.3. MLE

Instead of leveraging historical data via an ANCOVA model to improve estimation of the conditional mean $\widehat{\mu }$, TMLE takes the more direct approach of using the trial data in combination with a more flexible and possibly more accurate estimator of $\widehat{\mu }$. In general, TMLE uses estimates $\widehat{\mu }$ from machine learning models instead of linear regression. However, this creates another problem that must be solved.

3.3.1. The Problem With Machine‐Learning Plug‐In Estimators

Recall that for the ATE we are trying to estimate $\mathbb{E}\left[\mu \left(1,W\right)-\mu \left(0,W\right)\right]$. If we could generate accurate estimates $\widehat{\mu }\approx \mu$, via some kind of regression, a sensible estimator would be the “plug‐in” $\frac{1}{n}{\sum }_i\widehat{\mu }\left(1,W_i\right)-\widehat{\mu }\left(0,W_i\right)$, the difference of the “imputed” potential means for each subject. The accuracy of that estimate is dependent, however, on the degree to which $\widehat{\mu }\approx \mu$. It therefore behooves us to generate $\widehat{\mu }$ using powerful machine learning methods that can capture any nonlinearities, interactions, and so forth. that are present in $\mu$. The efficiency gains from TMLE (and similar strategies like double machine learning) come from using more accurate regression models.

Unfortunately, if $\widehat{\mu }$ comes from a generic machine learning model, the plug‐in estimator is generally too biased to allow for the construction of standard confidence intervals and p‐values. This happens because $\widehat{\mu }$ is optimized (e.g., via regularization and cross‐validation model selection) to minimize some kind of predictive metric like mean squared error, which is a combination of bias and variance. Therefore $\widehat{\mu }$ is typically biased even when machine learning is used and the bias does not reduce quickly enough with sample size to allow for standard inference. For rigorous statements on the plug‐in bias (using the von Mises expansion) one can look at [9, 28, 29], but we give an intuitive explanation below.

As a concrete example, consider fitting a lasso or ridge model for $\widehat{\mu }\left(a,w\right)={\alpha }_0+a{\alpha }_A+\mathit{x\alpha }$ in a randomized trial and imagine that the terms $x$ of the design matrix contain enough covariates and interactions that their linear combination can accurately model $\mu \left(a,w\right)$ in the population. Due to the presumably large number of terms in the design matrix, regularization is required and cross‐validation will be necessary to choose the appropriate level. If the design matrix is centered, the plug‐in estimate of the treatment effect will be $\frac{1}{n}\sum \widehat{\mu }\left(1,W_i\right)-\widehat{\mu }\left(0,W_i\right)={\widehat{\alpha }}_A$, the coefficient on treatment. But, due to the regularization, ${\widehat{\alpha }}_A$ will be shrunken (biased downwards) relative to the population value. Another way to think of this is that the graphs of the two curves $w\mapsto \widehat{\mu }\left(1,w\right)$ and $w\mapsto \widehat{\mu }\left(0,w\right)$ will be closer to each other than they should be.

To summarize and generalize from this example: all data‐adaptive, nonparametric models require some kind of regularization in order to prevent overfitting: they accept some bias in exchange for a reduction in variance. This bias is passed through to the plug‐in estimate of the treatment effect. Even if that bias disappears asymptotically (imagine decreasing the regularization as $n$ increases), for general machine learning schemes the bias often decreases more slowly than needed for it to be negligible when calculating confidence intervals and p‐values [29].

3.3.2. TMLE Debiases Plug‐Ins

TMLE solves this problem by “updating” the outcome prediction function $\widehat{\mu }$ to a new function ${\widehat{\mu }}^{*}$. The update step is done in such a way that ${\widehat{\mu }}^{*}$ is debiased for purposes of a plug‐in estimator. This can be thought of as a (targeted) “relaxation” of the implicit or explicit regularization used to fit $\widehat{\mu }$. As a result, TMLE allows us to collect the efficiency gains from using nonparametric regression models without paying any price in bias.

In the context of a simple trial with 1:1 randomization there is a version of the TMLE update step that is relatively easy to explain. The condition that any regression estimate $\widehat{\mu }$ would have to meet in order to be unbiased when used in a plug‐in estimator of the ATE in an RCT is

$$\mathbb{E}\left[\widehat{\mu }\left(1,W\right)-\widehat{\mu }\left(0,W\right)\right]=\mathbb{E}\left[Y\left(1\right)-Y\left(0\right)\right]=\mathbb{E}\left[Y|A=1\right]-\mathbb{E}\left[Y|A=0\right]$$ (6)

because by randomization $\mathbb{E}\left[Y\left(a\right)\right]=\mathbb{E}\left[Y|A=a\right]$. Since we cannot take exact expectations in the sample to calculate $\mathbb{E}\left[Y|A=1\right]-\mathbb{E}\left[Y|A=0\right]$ we replace these terms with sample averages

$$\underset{{\widehat{\mathrm{\Psi }}}_1^{*}}{\underbrace{\frac{1}{n_1}\sum _{A_i=1}{Y}_i}}-\underset{{\widehat{\mathrm{\Psi }}}_0^{*}}{\underbrace{\frac{1}{n_0}\sum _{A_i=1}{Y}_i}}$$

This is just the unadjusted estimate of the effect, which is known to be unbiased specifically in an RCT setting (but not in an observational study). In other words, in an RCT, we would like our TML estimate to be equal on average to the unadjusted estimate. This seems redundant but it is important because we will use the updated regression models to perform inference as well as to estimate the effect: that is where the gain in efficiency will come from. The point estimate will be the same (on average) as the unadjusted one, but the confidence intervals will generally be tighter because we will use the updated regression models to calculate the SE and reduce outcome variance that is explainable with covariates. The variance for TMLE is calculated the same way as shown in Equation (4) for the ANCOVA plug‐in estimator, but using the updated machine‐learning regression ${\widehat{\mu }}^{*}$, that is,

$${{\widehat{\sigma }}_{\infty }}^2=\frac{1}{n}\sum _{i=1}^n{\left(\frac{1_1(A_i)}{{\pi }_a}(Y-{\widehat{\mu }}^{*}(1,W_i))+({\widehat{\mu }}^{*}(1,W_i)-{\widehat{\Psi }}_1)-\frac{1_0(A_i)}{{\pi }_0}(Y-{\widehat{\mu }}^{*}(0,W_i))+({\widehat{\mu }}^{*}(0,W_i)-{\widehat{\Psi }}_0)\right)}^2.$$ (7)

This is the classical expression derived by Hahn [30]. Schuler [27] reformulate this expression (see their appendix A.2) in a way that makes it clear that the variance depends critically on the mean‐squared error of ${\widehat{\mu }}^{*}$. The (usually larger) standard error for the unadjusted estimate can also be calculated with the same formula, setting $\widehat{\mu }\left(a,W\right)$ equal to the average outcomes in each treatment arm, ${\widehat{\mathrm{\Psi }}}_a^{*}=\frac{1}{n_a}{\sum }_{A_i=a}{Y}_i$.

We have now motivated why we would like to update the outcome regression (to debias it) but we have not explained how to do so. Define $A_{\pm }=2A-1$ and $a_{\pm }=2a-1$, the treatment indicator mapped to $\left\{-1,1\right\}$. If we update the initial regression model using a single term

(8)

then one may algebraically verify that $\mathbb{E}\left[{\widehat{\mu }}^{*}\left(1,W\right)-{\widehat{\mu }}^{*}\left(0,W\right)\right]={\widehat{\mathrm{\Psi }}}_1^{*}-{\widehat{\mathrm{\Psi }}}_0^{*}$ as desired when $n_1=n_0=n/2$.

Recall that our problem is that bias from the regression estimate $\widehat{\mu }$ leaks into the plug‐in estimate $\frac{1}{n}\sum \widehat{\mu }\left(1,W\right)-\widehat{\mu }\left(0,W\right)$. That is, either $\widehat{\mu }\left(0,w\right)$ and $\widehat{\mu }\left(1,w\right)$ are too close together or too far apart due to whatever regularization is applied by the learning algorithm (in the LASSO example the regularization keeps them too close together and regularizes out the treatment effect). The effect of adding the update ${\mathrm{ϵ}}^{*}\left(2a-1\right)$ is to push the two curves apart by an amount $2{\mathrm{ϵ}}^{*}$ (or pull them together if $\mathrm{ϵ}$ is negative) while otherwise leaving their shape untouched. The quantity ${\mathrm{ϵ}}^{*}$ is engineered so that the resulting sample average distance between the two curves is exactly the unadjusted estimate of the treatment effect, as desired.

It is no coincidence that ${\mathrm{ϵ}}^{*}$ also happens to be the empirical maximum likelihood solution to the estimation problem $Y=\widehat{\mu }\left(A,W\right)+\mathrm{ϵ}{A}_{\pm }+\mathcal{N}\left(0,1\right)$ (verifying this again takes some algebra, see Appendix A). Until now, our explanation of TMLE has been informal, but this observation allows us to generalize and formalize to some extent. The model which we are searching with maximum likelihood in the described update is the set of conditional densities

$$\left\{p_{\mathrm{ϵ}}\left(Y|A,W\right)\sim \mathcal{N}\left(\widehat{\mu }\left(A,W\right)+\mathrm{ϵ}{A}_{\pm },1\right):\mathrm{ϵ}\in \mathrm{R}\right\}.$$

The fact that our updated regression is the MLE in this model is why the method is called targeted maximum likelihood (we will explain the “targeted” part shortly). We are doing maximum likelihood estimation in the “targeting” submodel shown above. In the context of such a model, let ${\mathbb{E}}_{\mathrm{ϵ}}\left[\cdot \right]={\mathbb{E}}_{p_{\mathrm{ϵ}}}\left[\cdot \right]$. The model shown above has the special properties that

1. the conditional mean at some value ${\mathrm{ϵ}}_0$ (in this case ${\mathrm{ϵ}}_0=0$) is ${\mathbb{E}}_{{\epsilon }_0}\left[Y|A,W\right]=\widehat{\mu }\left(A,W\right)$, the initial regression fit, and

2. the derivative of the log‐likelihood in $\mathrm{ϵ}$ is given by $\frac{d}{d\mathrm{ϵ}}\log p_{\mathrm{ϵ}}=A_{\pm }\left(Y-{\mu }_{\mathrm{ϵ}}\left(A,W\right)\right)$ where ${\mu }_{\mathrm{ϵ}}\left(A,W\right)={\mathrm{E}}_{\mathrm{ϵ}}\left[Y|A,W\right]$.

These properties are what make the model targeted for the debiasing of the ATE in an RCT.

There is not one single definition of what a TMLE is. However, in the context of an 1:1 RCT it has been shown that updating in any parametric model $\left\{p_{\mathrm{ϵ}}\left(Y|A,W\right):\mathrm{ϵ}\in {\mathrm{R}}^d\right\}$ that satisfies these two conditions will suffice for the TMLE update step in the sense of providing asymptotic semiparametric efficiency: one sets up the chosen model, finds the maximum likelihood estimate ${\mathrm{ϵ}}^{*}$ within it, and then takes ${\widehat{\mu }}^{*}\left(A,W\right)={\mathbb{E}}_{{\mathrm{ϵ}}^{*}}\left[Y|A,W\right]$ as the update. In this paper we are in the setting of a 1:1 RCT, and we will therefore consider this as our formal definition of TMLE for ATE estimation. There are more definitions of TMLEs in different contexts and for different estimands [10, 31].

Updating in a parametric model that satisfies these two conditions will make the resulting estimator be debiased the same way that our example was (using the additive normal update model) [9, 10]. When $\mathrm{ϵ}=\left[{\mathrm{ϵ}}_1,{\mathrm{ϵ}}_2,\dots {\mathrm{ϵ}}_d\right]$ is a multidimensional parameter, the second condition generalizes to $A_{\pm }\left(Y-\widehat{\mu }\left(A,W\right)\right)$ being contained in the linear span of the functions $\frac{\partial }{\partial {\mathrm{ϵ}}_j}\log p_{\mathrm{ϵ}}$. This happens in particular when there is some ${\mathrm{ϵ}}_j$ for which $\frac{\partial }{\partial {\mathrm{ϵ}}_j}\log p_{\mathrm{ϵ}}=A_{\pm }\left(Y-{\mu }_{\mathrm{ϵ}}\left(A,W\right)\right)$.

To summarize, TMLE constructs an updated outcome regression ${\widehat{\mu }}^{*}$ by maximizing the likelihood within a submodel chosen so that the update removes the bias relevant to the plug‐in using $\widehat{\mu }$ without targeting. We can summarize the procedure for ATE estimation in an RCT as:

1. Obtain an initial estimate $\widehat{\mu }$ for the conditional mean functions using a working outcome model, potentially some powerful machine learning method.

2. From $\widehat{\mu }$ obtain the predicted outcomes $\widehat{\mu }\left(A_i,W_i\right)$ for each participant in the study.

3. Use MLE with the trial data and predictions to estimate ${\mathrm{ϵ}}^{*}$ using the regression model $Y=\widehat{\mu }\left(A,W\right)+\mathrm{ϵ}{A}_{\pm }+\mathcal{N}\left(0,1\right)$.

4. Use ${\mathrm{ϵ}}^{*}$ to get the targeted predictions ${\widehat{\mu }}^{*}\left(0,W\right)=\widehat{\mu }\left(0,W\right)-{\mathrm{ϵ}}^{*}$ and ${\widehat{\mu }}^{*}\left(1,W\right)=\widehat{\mu }\left(1,W\right)+{\mathrm{ϵ}}^{*}$ for each participant in the study.

5. Average the targeted predictions to estimate the counterfactual mean ${\widehat{\mathrm{\Psi }}}_a^{*}$ for $a\in \left\{0,1\right\}$.

6. Contrast on the ATE scale, that is, ${\widehat{\mathrm{\Psi }}}_1^{*}-{\widehat{\mathrm{\Psi }}}_0^{*}$ and obtain the variance estimate using (7).

A complication to Steps 1 and 2 that we have so far not mentioned is that the initial outcome regression $\widehat{\mu }$ generally needs to be “cross‐fit” in order to assure $\sqrt{n}$‐consistency of the point estimate and consistency of the standard error [32, 33, 34, 35]. This allows valid inference without additional assumptions on the regression algorithms used. One way to do this is to use a leave‐one‐out estimation scheme, where the model is fit using $n-1$ participants to get the prediction of the one participant being left out of the fitting procedure. $v$‐fold cross estimation is also valid and more sensible for large datasets. Cross‐fit estimates of the outcome regression should be used to generate both the point estimate and the estimate of the standard error. See Wenjing and van der Laan [34] and Chernozhukov et al. [32] for mathematical justification and Zivich and Breskin [35] for a digestible description and visual depiction of cross‐fitting.

Besides TMLE, augmented inverse‐propensity weighting (AIPW) is another tool for debiasing the plug‐in based on an estimate $\widehat{\mu }$ from a machine learning model. AIPW is asymptotically equivalent to TMLE and they tend to behave very similarly. Many papers have examined the relationship between these different estimators, but this is not the focus of our current work [33, 35, 36].

There are some cases where TMLE (or AIPW) do not improve efficiency over ANCOVA. If the underlying data‐generating process is well‐specified by the linear model underlying the ANCOVA, then no efficiency gain is possible, and in finite samples ANCOVA will likely have an edge. Moreover, the comparison between the estimators is asymptotic, meaning that in small samples the relative variances may not match the limiting behaviors.

To conclude, machine‐learning plug‐in estimators improve over ANCOVA by allowing for data‐adaptive, possibly nonparametric regressions that explain away more of the variance in the outcome, often leading to increased power in estimating treatment effects. However, they incur bias due to regularization. TMLE (and AIPW) remove that bias, enabling us to construct asymptotic confidence intervals and pick up the power gains without suffering inflated type I error rates.

4. Within‐Trial Prognostic Score Adjustment

Next we show that constructing a prognostic score within the trial and using it in an ANCOVA model is equivalent to performing a TMLE update in an RCT.

Clinical trials are usually small, so using large external datasets to learn a prognostic score can be a good idea [4, 8, 19]. However, if external data are not available or trustworthy, one could imagine building the prognostic model on the trial data itself to eliminate the use of historical data. Specifically, one would fit a prognostic model on the trial data by regressing $Y$ onto $\left(A,W\right)$ with machine learning methods and then predict the outcome for each participant ${\widehat{\rho }}_1\left(A,W\right)=\widehat{\mu }\left(A,W\right)$, now predicting the outcome using the actual treatment allocation. The idea is then to add the predictions as an additional covariate in the design matrix for the ANCOVA method, following the prognostic score adjustment procedure, that is, using the plug‐in procedure described in Section 3. This procedure will be referred to as within‐trial prognostic score adjustment. This procedure can be summarized as:

1. Fit a prognostic model ${\widehat{\rho }}_1\left(A,W\right)=\widehat{\mu }\left(A,W\right)$ on the trial data using a working outcome model (potentially some powerful machine learning method).

2. From ${\widehat{\rho }}_1$ obtain the predicted outcomes ${\widehat{\rho }}_1\left(a,w\right)$ for each participant in the study.

3. Use MLE to estimate $\beta =\left({\beta }_1,{\beta }_2,{\beta }_3\right)$ using the regression model $Y={\beta }_1{A}_{\pm }+{\beta }_2\widehat{\mu }\left(A,W\right)+X{\beta }_3+\mathcal{N}\left(0,1\right)$. ³

4. Then follow Step 2–4 of the plug‐in method described in Section 3.1 to obtain the ATE and variance estimate.

Within‐trial adjustment is equivalent to TMLE with a specific submodel. To see this, consider a 1:1 randomized trial with the goal of estimating the ATE. In the TMLE procedure we first introduced we updated an initial outcome regression $\widehat{\mu }$ by finding the MLE ${\mathrm{ϵ}}^{*}$ in the model $Y=\mathrm{ϵ}{A}_{\pm }+\widehat{\mu }\left(A,W\right)+\mathcal{N}\left(0,1\right)$. However, we can choose another model to maximize over as long as the two conditions we stated above are satisfied (a common choice is a logistic regression update [15]). Another legitimate submodel is the linear regression $Y={\beta }_1{A}_{\pm }+{\beta }_2\widehat{\mu }\left(A,W\right)+X{\beta }_3+\mathcal{N}\left(0,1\right)$. The first condition is satisfied by $\beta =\left(\mathrm{0,1,0}\right)$ and the second by taking the derivative with respect to ${\beta }_1$.

The ANCOVA step of within‐trial prognostic adjustment is precisely this TMLE update, letting the in‐trial prognostic score ${\widehat{\rho }}_1$ play the role of the initial outcome regression $\widehat{\mu }$. Due to the equivalence between the robust standard error of the ANCOVA coefficient and the IF based standard error (Equation 7), the TMLE (IF) standard error also coincides with the within‐trial prognostic adjustment standard error from ANCOVA. Since $X$ are the terms in an arbitrary design matrix, the equivalence holds regardless of whether covariates besides the (within‐trial) prognostic score are adjusted for in the trial analysis.

5. Discussion

Since within‐trial prognostic adjustment is a TMLE, it is a consistent estimator of the ATE and inherits other beneficial properties (e.g., efficiency if the prognostic score models the conditional mean well). Within‐trial prognostic adjustment is therefore a valid method for increasing efficiency while eliminating the dependency on historical data in RCTs. However, outside of pedagogical contexts, it should simply be called TMLE instead of being re‐branded as “within‐trial prognostic adjustment.”

While within‐trial prognostic adjustment is a TMLE, if additional covariates besides the prognostic score are adjusted for, it is not the usual TMLE defined in Section 3.3 because it includes the covariates in the update submodel as linear predictors. However, for within‐trial prognostic adjustment, including these covariates in the update is redundant because the within‐trial prognostic scores should already capture the linear trends from the covariates. In other words, we should expect the covariate update coefficients to be near zero. Thus, including them in the update step is not only unnecessary but also has no meaningful effect. This is briefly demonstrated by a simulation study in Appendix B. So, while it is valid to use within‐trial prognostic score adjustment with additional covariates, there is no reason to reinvent the wheel. We therefore advocate for doing the regular TMLE already available in various software packages, which is equivalent to within‐trial prognostic adjustment for only the prognostic score. In terms of cross‐fitting, cross‐fitting the prognostic score in a within‐trial prognostic score adjustment is identical to cross‐fitting the outcome regression in TMLE. Therefore, cross‐fit, within‐trial prognostic adjustment is a standard cross‐fit TMLE [34].

Pre‐specification is important to bound type I error rates for all trial analyses and TMLE is no different [23, 37]. For TMLE and within‐trial prognostic score adjustment, what should be prespecified is the library of candidate models used for the outcome regression and the procedure for fitting, tuning, or selecting among them [38].

One could still consider including an additional prognostic score built from historical data to the TMLE or within‐trial procedures as proposed by Liao et al. [19]. Liao et al. [19] show that this can lead to further efficiency gains for small trials in particular. Specifically, we would expect to see a gain in efficiency when the historical data is similar to the new trial and when the amount of historical data is large compared to the new trial. Therefore, it is not either/or; you can combine historically‐built prognostic scores with TMLE (which is essentially the within‐trial procedure). The comparison of the within‐trial procedure and the standard prognostic score adjustment method has been examined in simulations by Elefsinioti et al. [39].

Funding

This work was supported by Innovationsfonden (2052‐00044B).

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A. Determining the TMLE Updating Step

In this section we show that ${\mathrm{ϵ}}^{*}$ in (8) is the maximum likelihood solution to the estimation problem $Y=\widehat{\mu }\left(A,W\right)+\mathrm{ϵ}{A}_{\pm }+\mathcal{N}\left(0,1\right)$. We define $U_i=Y_i-\widehat{\mu }\left(A_i,W_i\right)$ and $U={\left(U_1,U_2,\dots ,U_n\right)}^{\top }$. Then regressing $U$ on $A_{\pm }$ the design matrix $\mathbb{X}$ has rows $x=a_{\pm ,i}$. Thus, we have $\widehat{\epsilon }={\left({\mathbb{X}}^{\top }\mathbb{X}\right)}^{-1}{\mathbb{X}}^{\top }u$. First notice that ${\left({\mathbb{X}}^{\top }\mathbb{X}\right)}^{-1}=n^{-1}$ using $A_{\pm }$ instead of $A$. Therefore, we have

(A1)

Appendix B. Simulation Study

Here we examine the differences between within‐trial prognostic covariate adjustment and TMLE that may occur due to the within‐trial procedure including the covariates in the update submodel as linear predictors. This is done in different data generating scenarios and across different data set sizes. We simulate data based on the data generating process described in Højbjerre‐Frandsen et al. [8], but since we are only comparing the within‐trial procedure and TMLE we do not simulate historical data and therefore not distributional shifts. For completeness we will restate the data generating process and scenarios. The code is available here.

The data generating process for the current trial is represented in the structural causal model in (B1).

$$\begin{array}{l}{W}_1,W_2\sim \text{Unif}\left(-2,1\right)\\ W_3\sim \mathcal{N}\left(0,3\right)\\ W_4\sim \mathrm{Exp}\left(0.8\right)\\ W_5\sim \mathrm{\Gamma }\left(5,10\right)\\ W_6,W_7\sim \text{Unif}\left(1,2\right)\\ A\sim \text{Bern}\left(0.5\right)\\ Y\left(a\right)\mid W,U=m_a\left(W,U\right)+\mathcal{N}\left(\mathrm{0,1.3}\right)\\ Y=\mathit{AY}\left(1\right)+\left(1-A\right)Y\left(0\right).\end{array}$$ (B1)

In the homogeneous treatment effect scenario we have

$$m_0\left(W,U\right)=4.1\cdot \sin \left(W_2\right)+1.5\cdot I\left(W_4|>0.25\right)+1.5\cdot \sin \left(W_5\right)+1.4\cdot I\left(W_3|>2.5\right)$$

and $m_1\left(W,U\right)=\mathrm{ATE}+m_0\left(W,U\right).$ In the heterogeneous treatment effect scenario we use $m_0$ and,

$$\begin{array}{l}{m}_1\left(W,U\right)=4.3\cdot \sin {\left(W_2\right)}^2+1.3\cdot I\left(W_4|>0.25\right)+4.1\cdot I\left(W_2>0\right)\cdot \sin \left(W_5\right)\\ +1.6\cdot \sin \left(W_6\right)+1.4\cdot I\left(W_3|>2.5\right).\end{array}$$

In both scenarios $\mathrm{ATE}=0.84$.

We fix the current trial sample size $n$ = 200, and examine both the homogeneous and heterogeneous data generating scenario. We also examine the effect of varying the data set size by setting $n=\mathrm{50,60,70},\dots \mathrm{,200,225,250,275,300,325,350,375,400}$. For all scenarios, we have $N=250$ runs of the simulations. For estimation of the ATE we consider within‐trial prognostic score adjustment and TMLE. For the prognostic model estimation and the initial outcome model estimation for TMLE we use the Discrete Super Learner specified as in Højbjerre‐Frandsen et al. [8].

Figure B1 shows the SE estimates for the estimators in the different data generating scenarios. The filled dots represent the mean of the estimated SE and the stars represents the empirically estimated SE of the ATE estimates. Unsurprisingly, the performance of linear within‐trial prognostic score adjustment is similar to the TMLE across the two scenarios, but it is not identical as claimed in Section 5 due to the covariates being included in the submodel as linear predictors.

FIGURE B1.

Standard error estimates for different scenarios. Dots represent the mean of the estimated SE, while stars represent the empirically estimated SE of the ATE estimates across the 250 simulated pairs of datasets.

In Figure B2 the empirically estimated power and coverage is displayed across different data set sizes in the heterogeneous scenario. The TMLE method and within‐trial prognostic score adjustment align closely for all $n$ further supporting the claim in Section 5. The coverage is approximately 95% which is shown in Figure B2B.

FIGURE B2.

Empirical power and coverage with increasing sample size ($\tilde{n}=10n$). (A) Shows the empirically estimated power for different estimators as a function of the sample size (n). (B) Displays the empirically estimated coverage for the same estimators across varying sample sizes with a significance level of 5% (dotted black line at 95%).

Data Availability Statement

The code that generated the data that support the findings of this study (see Appendix B) is openly available at https://github.com/NNEHFD/within_trial_prog.