A Targeted Maximum Likelihood Estimator of a Causal Effect on a Bounded Continuous Outcome

Abstract

Targeted maximum likelihood estimation of a parameter of a data generating distribution, known to be an element of a semi-parametric model, involves constructing a parametric model through an initial density estimator with parameter ɛ representing an amount of fluctuation of the initial density estimator, where the score of this fluctuation model at ɛ = 0 equals the efficient influence curve/canonical gradient. The latter constraint can be satisfied by many parametric fluctuation models since it represents only a local constraint of its behavior at zero fluctuation. However, it is very important that the fluctuations stay within the semi-parametric model for the observed data distribution, even if the parameter can be defined on fluctuations that fall outside the assumed observed data model. In particular, in the context of sparse data, by which we mean situations where the Fisher information is low, a violation of this property can heavily affect the performance of the estimator. This paper presents a fluctuation approach that guarantees the fluctuated density estimator remains inside the bounds of the data model. We demonstrate this in the context of estimation of a causal effect of a binary treatment on a continuous outcome that is bounded. It results in a targeted maximum likelihood estimator that inherently respects known bounds, and consequently is more robust in sparse data situations than the targeted MLE using a naive fluctuation model.

When an estimation procedure incorporates weights, observations having large weights relative to the rest heavily influence the point estimate and inflate the variance. Truncating these weights is a common approach to reducing the variance, but it can also introduce bias into the estimate. We present an alternative targeted maximum likelihood estimation (TMLE) approach that dampens the effect of these heavily weighted observations. As a substitution estimator, TMLE respects the global constraints of the observed data model. For example, when outcomes are binary, a fluctuation of an initial density estimate on the logit scale constrains predicted probabilities to be between 0 and 1. This inherent enforcement of bounds has been extended to continuous outcomes. Simulation study results indicate that this approach is on a par with, and many times superior to, fluctuating on the linear scale, and in particular is more robust when there is sparsity in the data.

Erratum

On September 20, 2010 the following acknowledgment was added for the article:

“This work was supported by NIH grant R01AI074345-04.”

1. Introduction

Targeted maximum likelihood estimation (TMLE) yields semi-parametric efficient substitution estimators of parameters in semi-parametric models (van der Laan and Rubin, 2006). In particular, it can be applied to estimating the statistical counterpart of a causal parameter. In this article a new targeted maximum likelihood estimator for estimating a causal effect of a binary treatment on a continuous outcome is introduced. This estimator is more robust than a previously presented TMLE procedure when there is sparsity in the data that decreases the identifiability of the parameter of interest.

Sparsity is defined as low information in the dataset for the purpose of learning the target parameter. Formally, the Fisher information, I, is defined as sample size n divided by the variance of the efficient influence curve: I = n/var(D* (O)), where D* (O) is the efficient influence curve of the target parameter at the true data generating distribution. The reciprocal of the variance of the efficient influence curve can be viewed as the information one observation contains for the purpose of learning the target parameter. Since the variance of the efficient influence curve divided by n times the variance of an asymptotically efficient estimator converges to 1 when the sample size converges to infinity, one can also think of the information I as the reciprocal of the variance of an efficient estimator of the target parameter. Thus, sparsity with respect to a particular target parameter corresponds with small sample size relative to the variance of the efficient influence curve for that target parameter.

Section 2 of the paper provides background on the application of TMLE methodology in the context of sparsity, and its power relative to other semi-parametric efficient estimators by being a substitution estimator respecting global constraints of the semi-parametric model. Even though an estimator can be asymptotically efficient without utilizing global constraints, the global constraints are instrumental in the context of sparsity with respect to the target parameter, motivating the need for semi-parametric efficient substitution estimators, and for a careful choice of fluctuation function for the targeted MLE step that fully respects these global constraints. A rigorous demonstration of the proposed targeted MLE of the causal effect of a binary treatment on a bounded continuous outcome follows, and it is contrasted to a targeted MLE that makes use a fluctuation function that does not respect the bounds.

Simulation studies described in Section 3 compare the new TMLE estimator of the causal effect, which relies on a logistic fluctuation of an initial density estimate, with the traditional TMLE estimator, with and without sparsity in the data. Results for other commonly applied estimators, the inverse-probability-of-treatment weighted estimator (IPTW) (Hernan et al., 2000; Robins, 2000b), a double robust augmented IPTW estimator (aug-IPTW) (Robins and Rotnitzky, 2001; Robins et al., 2000; Robins, 2000a) that is efficient but not a substitution estimator, and the maximum likelihood substitution estimator according to a parametric model (MLE) (Robins, 1986) are also presented.

2. TMLE for causal effect estimation on a continuous outcome

The targeted MLE is a semi-parametric efficient substitution estimator of a target parameter Ψ(P₀) of a true distribution P₀ ∈ 𝒨, based on sampling n i.i.d. O₁, …, O_n from P₀. Here P₀ is known to be an element of a semi-parametric statistical model 𝒨. We will start with providing a succinct summary of how it works. For more details we refer to our articles on this topic (van der Laan et al., 2009).

Firstly, one notes that Ψ(P₀) = Ψ(Q₀) only depends on P₀ through a relevant part Q₀ = Q(P₀) of P₀. Secondly, one proposes a loss function L(Q)(O) so that Q₀ = arg min_Q∈𝒬 E₀L(Q)(O), where 𝒬 = {Q(P) : P ∈ 𝒨}. Thirdly, one uses minimum loss-based learning, such as super learning (van der Laan et al., 2007), fully utilizing the power and optimality results for loss-based cross-validation to select among candidate estimators, to obtain an initial estimator $Q_n^0$ of Q₀. Fourthly, one proposes a parametric fluctuation $Q_{ng}^0(\epsilon )$, possibly indexed by nuisance parameter g₀ = g(P₀), so that

$${\frac{d}{d\epsilon }L(Q_{ng}^0(\epsilon ))(O)|}_{\epsilon =0}=D*(Q_n^0,\hspace{0.17em}g)\hspace{0.17em}(O),$$ (1)

where D* (Q₀, g₀) is the canonical gradient/efficient influence curve of Ψ : 𝒨 → ℝ at P₀. Fifthly, one computes the amount of fluctuation

$${\epsilon }_n=\text{arg}\hspace{0.17em}\underset{\epsilon }{\text{min}}\sum _{i=1}^{n}L(Q_{n{g}_n}^0(\epsilon ))\hspace{0.17em}(O_i),$$

where g_n is an estimator of the unknown nuisance parameter g₀. This yields an update $Q_n^1=Q_{{ng}_n}^0({\epsilon }_n)$. This updating of an initial estimator $Q_n^0$ into a next $Q_n^1$ is iterated till convergence resulting in a $Q_n^{*}$. Since at the last step the amount of fluctuation ε_n ≈ 0, this final $Q_n^{*}$ will solve the efficient influence curve estimating equation

$$0=\sum _{i=1}^{n}D*(Q_n^{*},\hspace{0.17em}{g}_n)\hspace{0.17em}(O_i),$$

representing a fundamental ingredient for establishing asymptotic efficiency of $\Psi (Q_n^{*})$: recall that an estimator is efficient if and only if it is asymptotically linear with influence curve equal to the efficient influence curve D* (Q₀, g₀). Finally, the targeted MLE of ψ₀ is the substitution estimator $\Psi (Q_n^{*})$.

Thus we see that the targeted MLE involves constructing a parametric model $Q_n^0(\epsilon )$ through the initial estimator $Q_n^0$ with parameter ε representing an amount of fluctuation of the initial estimator, where the score of this fluctuation model at ε = 0 equals the efficient influence curve. The latter constraint can be satisfied by many parametric models, since it represents only a local constraint of its behavior at zero fluctuation. However, it is very important that the fluctuations stay within the model for the observed data distribution, even if the parameter can be defined on fluctuations that fall outside the assumed observed data model. In particular, in the context of sparse data, a violation of this property can heavily affect the performance of the estimator.

One important strength of the semi-parametric efficient targeted MLE relative to the alternative semi-parametric efficient estimating equation methodology (van der Laan and Robins, 2003) is that it does respect the global constraints of the observed data model since it is a substitution estimator $\Psi (Q_n^{*})$ with $Q_n^{*}$ an estimator of a relevant part Q₀ of the true distribution of the data in the observed data model. The estimating equation methodology does not result in substitution estimators and thereby often ignores important global constraints of the observed data model, though Tan (2008) introduces a non-parametric likelihood based approach to constructing a double robust estimator that is not a substitution estimator, and offers a comparison with other estimators, including TMLE that is not constrained to remain within the bounds of the observed data model. Ignoring constraints comes at a price in the context of sparsity. Indeed, simulations have confirmed this gain of targeted MLE relative to the efficient estimating equation method in the context of sparsity (Stitelman and van der Laan, 2010), and it is again demonstrated in this article. However, if the targeted MLE starts violating this principle of being a substitution estimator by allowing $Q_n^{*}$ to fall outside the assumed observed data model, this advantage is compromised. Therefore, it is crucial that a fluctuation model is used that is guaranteed to stay within the wished observed data model.

To demonstrate this important consideration of selecting a valid fluctuation model in the construction of targeted MLE, we consider the problem of estimating a causal effect of a binary treatment A on a continuous outcome Y, based on observing n i.i.d. copies of O = (W, A, Y) ∼ P₀, where W is the set of confounders. Under non-parametric structural equation model (NPSEM) W = f_W (U_W), A = f_A (W, U_A), Y = f_Y (W, A, U_Y) with a structure on the exogenous variables U = (U_W, U_A, U_Y) satisfying the no unmeasured confounders assumption (A ⊥ Y (a) | W for the counterfactuals Y (a) defined by this NPSEM), the additive causal effect E(Y (1) – Y (0)) can be identified from the observed data distribution through the following statistical parameter of P₀:

$$\Psi (P_0)=E_0(E_0(Y|A=1,\hspace{0.17em}W)-E_0(Y|A=0,\hspace{0.17em}W)).$$

Suppose that it is known that Y ∈ [a, b] for some a < b. Alternatively, one might have truncated the original data to fall in such an interval and focus on the causal effect of treatment on this truncated outcome, motivated by the fact that estimating conditional means of unbounded, or very heavy tailed, outcomes requires very large data sets.

Let Y* = (Y – a)/(b – a) be the linearly transformed outcome within [0, 1], and define

$$\Psi *(P_0)=E_0(E_0(Y*|A=1,\hspace{0.17em}W)-E_0(Y*|A=0,\hspace{0.17em}W)).$$

We note that

$$\Psi (P_0)=(b-a)\Psi *(P_0).$$

An estimate, limit distribution, and confidence interval for Ψ* (P₀) is now immediately mapped into an estimate, limit distribution, and confidence interval for Ψ(P₀), by simple multiplication by (b – a). As a consequence, without loss of generality, we can assume a = 0 and b = 1 so that Y ∈ [0, 1].

The efficient influence curve of the statistical parameter Ψ : 𝒨 → ℝ, defined on a non-parametric statistical model 𝒨 for P₀, at the true distribution P₀, is given by

$$D*(P_0)=\frac{2A-1}{g_0(A|W)}(Y-{\overline{Q}}_0(W,\hspace{0.17em}A))+{\overline{Q}}_0(W,\hspace{0.17em}1)-{\overline{Q}}_0(W,\hspace{0.17em}0)-\Psi (Q_0),$$

where Q̄₀(W, A) = E₀(Y | A, W), and Q₀ = (Q_W, Q̄₀) denotes both this conditional mean Q̄₀ as well as the marginal distribution Q_W of W. Note that indeed Ψ(P₀) only depends on P₀ through Q̄₀ and the marginal distribution of W. We will use the notation Ψ(P₀) and Ψ(Q₀) interchangeably.

We will now define a targeted MLE of Ψ(Q₀) as follows. Let ${\overline{Q}}_n^0$ be an initial estimator of Q̄₀(W, A) = E(Y | A, W) with predicted values in (0, 1). In addition, we estimate P_W with the empirical distribution of W. Let $Q_n^0$ denote the resulting initial estimator of Q₀. The targeted MLE step will also require an estimator g_n of g₀ = P_A|W. Only the conditional mean ${\overline{Q}}_n^0$ will be modified by the targeted MLE procedure defined below: this makes sense since the empirical distribution of W is already a non-parametric maximum likelihood estimator so that no bias gain with respect to the target parameter will be obtained by modifying it.

We can represent the estimator ${\overline{Q}}_n^0$ as ${\overline{Q}}_n^0=\frac{1}{1+\text{exp}(-f_n^0)}$ with $f_{\text{n}}^0=\text{log}({\overline{Q}}_n^0/(1-{\overline{Q}}_n^0))$. Consider now the fluctuation model

$${\overline{Q}}_n^0(\epsilon )=\frac{1}{1+\text{exp}(-\{f_n^0+\epsilon h\})},$$

with parameter ε, indexed by a function

$$h(g_n)\hspace{0.17em}(W,\hspace{0.17em}A)=\frac{2A-1}{g_n\hspace{0.17em}(A|W)}.$$

Equivalently, we can write this as $\text{logit}{\overline{Q}}_n^0(\epsilon )=\text{logit}{\overline{Q}}_n^0+\epsilon h(g_n)$.

Consider now the following loss function for Q̄₀:

$$-L(\overline{Q})\hspace{0.17em}(O)=Y\hspace{0.17em}\text{log}\overline{Q}(W,\hspace{0.17em}A)+(1-Y)\text{log}(1-\overline{Q}(W,\hspace{0.17em}A)).$$

Note that this is the log-likelihood of the conditional distribution of a binary outcome Y, but now extended to continuous outcomes in [0, 1]. (See also Wedderburn (1974), McCullagh (1983) for earlier use of logistic regression for continuous outcomes.) It is thus known that this loss function is a valid loss function for the conditional distribution of a binary Y, but we need that it is a valid loss function for a conditional mean of a continuous Y ∈ [0, 1]. We have the following lemma establishing this result about this loss function.

Lemma 1 We have that

$${\overline{Q}}_0=\underset{\overline{Q}}{\text{argmin}}{E}_{0}L(\overline{Q}),$$

where the minimum is taken over all functions of (W, A) which map into (0, 1). In addition, define the fluctuation function

$$\mathit{\text{logit}}\overline{Q}(\epsilon )=\mathit{\text{logit}}\overline{Q}+\epsilon h.$$

For any function h we have

$${\frac{d}{d\epsilon }L(\overline{Q}(\epsilon ))|}_{\epsilon =0}=h(W,\hspace{0.17em}A)\hspace{0.17em}(Y-\overline{Q}(W,\hspace{0.17em}A)).$$

Proof: Let Q̄₁ be a local minimum and consider the fluctuation Q̄₁(ε) defined above. Then the derivative of E₀L(Q̄₁ (ε)) at ε = 0 equals zero. However,

$$-{\frac{d}{d\epsilon }L({\overline{Q}}_1(\epsilon ))|}_{\epsilon =0}=h(W,\hspace{0.17em}A)\hspace{0.17em}(Y-{\overline{Q}}_1(W,\hspace{0.17em}A)).$$

Thus, it follows that

$$E_{0}h(W,\hspace{0.17em}A)\hspace{0.17em}(Y-{\overline{Q}}_1(W,\hspace{0.17em}A))=E_{0}h(W,\hspace{0.17em}A)\hspace{0.17em}({\overline{Q}}_0-{\overline{Q}}_1)\hspace{0.17em}(W,\hspace{0.17em}A).$$

But this needs to hold for any function h(W, A), which proves that Q̄₁ = Q̄₀ a.e.

This proves that L(Q̄) is a valid loss function for the conditional mean Q̄₀. Indeed, we can use L(Q̄) as loss function to construct an initial estimator of Q̄₀, and or use cross-validation to select among candidate targeted maximum likelihood estimators, such as in the collaborative targeted MLE procedure. For the purpose of construction of an initial estimator one could also use a minimum loss-based super learner based on the squared error loss function L₂(Q̄) = (Y – Q̄(W, A))², possibly with weights.

Given an initial estimator ${\overline{Q}}_n^0$, and our proposed fluctuation function ${\overline{Q}}_n^0(\epsilon )$, we have

$${\frac{d}{d\epsilon }L({\overline{Q}}_n^0(\epsilon ))|}_{\epsilon =0}=h(g)\hspace{0.17em}(W,\hspace{0.17em}A)\hspace{0.17em}(Y-{\overline{Q}}_n^0(W,\hspace{0.17em}A)),$$

giving us the wished first component $D_1^{*}$ of the efficient influence curve $D*=D_1^{*}+D_2^{*}$.

Let’s use the log-likelihood loss function, −logQ_W, as loss function for the marginal distribution of W, so that our combined loss function is given by L(Q) = −logQ_W + L(Q̄). In addition, we use as fluctuation of the empirical distribution Q_Wn, $Q_{W_n}({\epsilon }_1)=(1+{\epsilon }_1{D}_2^{*}(Q))Q_{W_n}$, where $D_2^{*}(Q)=\overline{Q}(W,\hspace{0.17em}1)-\overline{Q}(W,\hspace{0.17em}0)-\Psi (Q)$ is the remaining component of the efficient influence curve. With these choices we indeed now have that

$${\frac{d}{d\epsilon }L(Q(\epsilon ))|}_{\epsilon =0}=D*(Q,\hspace{0.17em}g).$$

This shows that we succeeded in defining a loss function for Q₀ = (Q_W, Q̄₀) and fluctuation function so that the wished derivative (1) indeed yields the efficient influence curve.

The MLE of ε₁ equals zero, so that the update of Q_Wn equals Q_Wn itself. The empirical mean of the component $D_2^{*}=\overline{Q}(W,\hspace{0.17em}1)-\overline{Q}(W,\hspace{0.17em}0)-\Psi (Q)$ of the efficient influence curve is always equal to zero, due to the fact that we estimate the marginal distribution of W with the empirical distribution of W.

The amount of fluctuation of ε for fluctuating ${\overline{Q}}_n^0$ is given by

$${\epsilon }_n^0=\underset{\epsilon }{\text{argmin}}{P}_{n}L({\overline{Q}}_n^0(\epsilon )).$$

This “maximum likelihood” estimator of ε can be computed with generalized linear regression using the binomial link, i.e. the logistic regression MLE procedure, simply ignoring that the outcome is not binary, which also corresponds with iterative re-weighted least squares estimation using weights 1/Q̄(1 – Q̄).

This provides us with the targeted MLE update $Q_n^1=Q_n^0({\epsilon }_n^0)$, where the empirical distribution of W did not get updated, and ${\overline{Q}}_n^0$ did get updated as ${\overline{Q}}_n^0({\epsilon }_n^0)$. Iterating this procedure now defines the targeted MLE $Q_n^{*}$, but as in the binary outcome case, we have that $Q_n^2=Q_n^1({\epsilon }_n^1)=Q_n^1$ since the next MLE ${\epsilon }_n^1=0$. Thus convergence occurs in one step, so that $Q_n^{*}=Q_n^1$. The targeted MLE of ψ₀ is thus given by $\Psi (Q_n^{*})=\Psi (Q_n^1)$. As predicted, we have that the targeted MLE $Q_n^{*}$ solves the efficient influence curve estimating equation $P_{n}D*(Q_n^{*},\hspace{0.17em}{g}_n,\hspace{0.17em}\Psi (Q_n^{*}))=0$.

An inspection of this efficient influence curve,

$$D*(P_0)=\frac{2A-1}{g_0(A|W)}(Y-{\overline{Q}}_0(W,\hspace{0.17em}A))+{\overline{Q}}_0(W,\hspace{0.17em}1)-{\overline{Q}}_0(W,\hspace{0.17em}0)-\Psi (Q_0),$$

reveals that there are two potential sources of sparsity. Small values for g₀(A | W) and large outlying values of Y inflate the variance. Enforcing (e.g., known) bounds on Y and g₀ in the estimation procedure provides a means for controlling these sources of variance. We note that, even if there is strong confounding causing some large values of $h_{g_n^0}$, the resulting targeted MLE ${\overline{Q}}_n^{*}$ remains bounded in (0, 1), so that the targeted MLE $\Psi (Q_n^{*})$ fully respects the global constraints of the observed data model. On the other hand, the augmented IPTW estimator obtained by solving $P_{n}D*(Q_n^0,\hspace{0.17em}{g}_n,\hspace{0.17em}\psi )=0$ in ψ yields the estimator

$${\psi }_n=\frac{1}{n}\sum _{i=1}^n{h}_{g_n}^0(W_i,\hspace{0.17em}{A}_i)\hspace{0.17em}(Y_i-{\overline{Q}}_n^0(W_i,\hspace{0.17em}{A}_i))+{\overline{Q}}_n^0(W_i,\hspace{0.17em}1)-{\overline{Q}}_n^0(W_i,\hspace{0.17em}0),$$

which can easily fall outside [0, 1] if for some observations W_i, g_n(1 | W_i) is close to 1 or 0. This represents the price of not being a substitution estimator.

Contrasting with targeted MLE using linear fluctuation function. Alternatively, we would employ the targeted MLE using the L₂(Q̄) = (Y – Q̄(W, A))² loss function, and fluctuation function Q̄⁰(ε) = Q̄⁰ + εh(g), so that (1) is still satisfied. In this case, large values of h(g) will result in predicted values of Q̄⁰(ε_n) that are out of the bounds [a, b]. Therefore, this version of targeted MLE is not respecting the global constraints of the model, i.e., the knowledge that Y ∈ [a, b]. A comparison based on simulated data of the targeted MLE using the logistic fluctuation function and the targeted MLE using this linear fluctuation function is provided in the next section.

3. Simulation studies for the additive effect of a binary point treatment on a continuous outcome

Two simulation studies illustrate the effects of employing a logistic vs. linear fluctuation on TMLE estimator performance with and without sparsity in the data, where a high degree of sparsity corresponds to a target parameter that is borderline-identifiable. As above, the parameter of interest is defined as the marginal effect of a binary point treatment on the outcome, ψ₀ = E_W (E(Y | A = 1, W) – E(Y | A = 0, W)).

The “traditional” targeted maximum likelihood approach to estimating an additive treatment effect when the outcome is continuous is to fluctuate the initial density estimate on a linear scale. Given ${\overline{Q}}_n^0(W,\hspace{0.17em}A)$, an initial estimate of the conditional mean of Y given (W, A), the fluctuation function is defined as ${\overline{Q}}_n^0(\epsilon )={\overline{Q}}_n^0+\epsilon (h_{g_n})$ and the loss function L(Q̄) is chosen to be the squared error loss function, so that we still have the required constraint (1). The estimate ε_n can be obtained by estimating ε with a linear regression of Y on h_gn, using the initial fit, ${\overline{Q}}_n^0(W,\hspace{0.17em}A)$, as offset.

A second TMLE estimate using the logistic fluctuation method described in Section 2 is also obtained. Y is transformed into Y* ∈ [0, 1] by shifting and scaling the values. In the simulation setting, Y is not bounded, so that we do not have an a priori a and b bound on Y. Instead of truncating Y and redefining the target parameter as the causal effect on the truncated Y, we still aim to estimate the causal effect on the original Y. Therefore, we set a = min(Y), b = max(Y), and

$$Y*=\frac{Y-a}{b-a}.$$

An initial estimate, ${\overline{Q}}_n^{0,Y*}(W,\hspace{0.17em}A)=E(Y*|W,\hspace{0.17em}A)$, is obtained, and then represented as a logistic function of its logit-transformation. Note that logit(x) is not defined when x = 0 or 1, therefore in practice ${\overline{Q}}_n^{0,Y*}(W,\hspace{0.17em}A)$ is bounded away from 0 and 1 by truncating it at (α, (1 – α)). We used α = 0.005 in these simulation studies, which did not yield appreciably different results than setting α = 0.001 or α = 0.01. The function ${\overline{Q}}_n^{0,Y*}$ is fluctuated on the logit scale with $\text{logit}{\overline{Q}}_n^{0,Y*}(\epsilon )=\text{logit}{\overline{Q}}_n^{0,Y*}+\epsilon h(g_n)$, using the same clever covariate, h_gn (W, A), employed in the linear fluctuation described above. Fitting ε is again carried out using standard software, but this time using logistic regression of Y* on h_gn (W, A) with offset $\mathit{\text{logit}}(Q_n^{0,Y*}(W,\hspace{0.17em}A))$. This results in the updated ${\overline{Q}}_n^{1,Y*}$. Fitted values for ${\overline{Q}}_n^{1,Y*}(W,\hspace{0.17em}A)$ are mapped back to the original scale: ${\overline{Q}}_n^{1,Y}={\overline{Q}}_n^{1,Y*}(W,\hspace{0.17em}A)*(b-a)+a$. The marginal distribution is estimated with the empirical distribution of W, giving the $Q_n^{*}=Q_n^1=(Q_{W,n},\hspace{0.17em}{\overline{Q}}_n^{1,Y})$ of (Q_W, Q̄₀). The estimate

$${\psi }_n=\Psi (Q_n^{*})=\frac{1}{n}\sum _{i=1}^n{\overline{Q}}_n^{1,Y}(W_i,\hspace{0.17em}1)-{\overline{Q}}_n^{1,Y}(W_i,\hspace{0.17em}0)$$

is the targeted MLE of the wished additive causal effect ψ₀.

Parameter estimates were also obtained using the augmented inverse probability of treatment weighed estimator (aug-IPTW),

$$\begin{array}{lll}{\psi }_n^{aug-IPTW}\hfill & =\hfill & \frac{1}{n}\sum _{i=1}^n\frac{2A-1}{g_n(A_i|W_i)}(Y_i-{\overline{Q}}_n^0(W_i,\hspace{0.17em}{A}_i))\hfill \\ \hfill & \hfill & +\frac{1}{n}\sum _{i=1}^n({\overline{Q}}_n^0(W_i,\hspace{0.17em}1)-{\overline{Q}}_n^0(W_i,\hspace{0.17em}0)).\hfill \end{array}$$

Both the targeted MLE and the augmented IPTW estimator are double robust so that these estimators will be consistent for ψ₀ if either g_n or ${\overline{Q}}_n^0$ is consistent for g₀ and Q̄₀, respectively. Both the targeted MLE and the augmented IPTW estimator are asymptotically efficient if both g_n and ${\overline{Q}}_n^0$ are consistent.

In this simulation study we will use simple parametric MLE’s as initial estimators ${\overline{Q}}_n^0$ and g_n, even though we recommend the utilization of super learning in practice. The purpose of this simulation is to investigate the performance of the updating step under misspecified and correctly specified ${\overline{Q}}_n^0$, and for that purpose we can work with parametric MLE fits.

Results from two estimation methods that are not double robust and semi-parametric efficient are included as well. The maximum likelihood estimator according to a parametric model for Q̄₀ (MLE), used as initial estimator in the targeted MLE and augmented IPTW, is included for the sake of evaluating the bias reduction step carried out by these two semi-parametric efficient procedures. Inverse probability of treatment weighted (IPTW) estimators are consistent when g_n(A | W) is a consistent estimator of the treatment mechanism g₀(A | W) = P (A = 1|W), but are known to be inefficient. These two estimators are defined as

$$\begin{array}{l}{\psi }_n^{MLE}=\frac{1}{n}\sum _{i-1}^n{\overline{Q}}_n^0(W_i-1)-{\overline{Q}}_n^0(W_i,\hspace{0.17em}0),\\ {\psi }_n^{IPTW}=\frac{1}{n}\sum _{i=1}^n(2A-1)\frac{Y_i}{g_n(A_i|W_i)}.\end{array}$$

3.1. Data generation

Covariates W₁, W₂, W₃ were generated as independent binary random variables,

$$W_1,\hspace{0.17em}{W}_2,\hspace{0.17em}{W}_3\sim \mathit{\text{Bernoulli}}(0.5).$$

Two treatment mechanisms were defined that differ only in the values of the coefficients for each covariate:

$$g_0(1|W)=P(A=1|W)={\text{logit}}^{-1}(a{W}_1+b{W}_2+c{W}_3).$$

We consider two settings:

$$a_1=0.5,\hspace{0.17em}{b}_1=1.5,\hspace{0.17em}{c}_1=-1\hspace{0.17em}\text{and}\hspace{0.17em}{a}_2=1.5,\hspace{0.17em}{b}_2=4.5,\hspace{0.17em}{c}_2=-3.$$

We refer to these two treatment mechanisms as g_0,1 and g_0,2, respectively. The observed outcome Y was generated as

$$\begin{array}{ccc}\hfill Y& =\hfill & {\overline{Q}}_0(W,\hspace{0.17em}A)+e,\hspace{0.17em}e\sim N(0,\hspace{0.17em}1),\hfill \\ \hfill {\overline{Q}}_0(W,\hspace{0.17em}A)& =\hfill & A_j+2W_1+3W_2-4W_3.\hfill \end{array}$$

For both simulations the true additive causal effect equals one: ψ₀ = 1. In both simulations predicted values for g_n(A | W) are bounded away from 0 and 1 by truncating at (p, (1 – p)), with p = 0.01. Treatment assignment probabilities based on mechanism g_0,1 range from 0.269 to 0.881, indicating no sparsity in the data for simulation 1. In contrast, simulation 2 poses a challenging estimation problem in the context of sparse data. Treatment assignment probabilities based on mechanism g_0,2 range from 0.047 to 0.998. These extreme values are nevertheless not uncommon for data from observational studies (see for example Dehejia and Wahba (2002); Stukel et al. (2007)).

Estimates were obtained for 1000 samples of size n = 1000 from each data generating distribution. Treatment assignment probabilities, g₀(A | W), were estimated using a correctly specified logistic regression model. A correctly specified main terms regression model was used to obtain ${\overline{Q}}_{\mathit{\text{cor}}}^0(W,\hspace{0.17em}A)$. In addition, a misspecified initial estimate, ${\overline{Q}}_{\mathit{\text{mis}}}^0(W,\hspace{0.17em}A)$, was obtained by regressing Y on A.

We expect MLE estimates based on ${\overline{Q}}_{\mathit{\text{cor}}}^0$ to be unbiased and efficient, while those based on ${\overline{Q}}_{\mathit{\text{mis}}}^0$ will be biased. IPTW estimates only depend on consistent estimation of g₀, thus are identical regardless of how Q̄₀ is estimated. For both simulations g_n is a consistent estimator, thus it is reasonable to expect unbiased IPTW estimates, with more variation in simulation 2 estimates. The targeted MLE and the augmented IPTW are known to be unbiased if g_n is consistent, and asymptotically efficient when both Q̄₀ and g₀ are consistently estimated. Though correctly estimating g₀ will asymptotically correct for any bias due to mis-specification of ${\overline{Q}}_n^0$, this is not guaranteed in finite samples, especially when there is sparsity. For simulation 2 we expect TMLE_log, using the logistic fluctuation, to outperform TMLE_lin, using the linear fluctuation.

3.2. Results

Table 1 reports the average estimate, bias, empirical variance, and mean squared error (MSE) for each estimator, under different specifications of the initial estimator ${\overline{Q}}_n^0$. In all cases g_n is consistent, and bounded at (0.01, 0.99). In simulation 1, when Q̄₀ is correctly estimated all estimators perform quite well, though as expected, IPTW is the least efficient. However, when Q̄₀ is incorrectly estimated, the MLE estimator is biased and has high variance relative to the other estimators. Because g_n(A | W) is correctly specified, IPTW and aug-IPTW provide unbiased estimates, as do both TMLEs. TMLE_log is on a par with TMLE_lin, as there is no sparsity in the data, and both are more efficient than any of the other estimators.

Table 1:

Estimator performance for simulations 1 and 2 when the initial estimator of Q̄₀ is correct and misspecified. Results are based on 1000 samples of size n = 1000.

	Q̄0correctly estimated				Q̄0incorrectly estimated
	ave	bias	var	MSE	ave	bias	var	MSE
Simulation 1
MLE	1.003	0.003	0.005	0.005	3.075	2.075	0.030	4.336
IPTW	1.006	0.006	0.009	0.009	1.006	0.006	0.009	0.009
aug-IPTW	1.003	0.003	0.005	0.005	1.005	0.005	0.010	0.010
TMLElog	0.993	−0.007	0.005	0.005	0.993	−0.007	0.006	0.006
TMLElin	0.993	−0.007	0.005	0.005	0.993	−0.007	0.006	0.006
Simulation 2
MLE	1.001	0.001	0.009	0.009	4.653	3.653	0.025	13.370
IPTW	1.554	0.554	0.179	0.485	1.554	0.554	0.179	0.485
aug-IPTW	0.999	−0.001	0.023	0.023	1.708	0.708	0.298	0.798
TMLElog	0.989	−0.011	0.037	0.037	0.722	−0.278	0.214	0.291
TMLElin	0.986	−0.014	0.042	0.042	−0.263	−1.263	2.581	4.173

In simulation 2 all estimators except IPTW are unbiased when Q̄₀ is correctly estimated. In this case, both TMLE estimators have higher variance than aug-IPTW, and all three are more efficient than IPTW, but less efficient than the parametric MLE estimator. Though asymptotically the IPTW estimator is expected to be unbiased in this simulation, since g_n is a consistent estimator of g₀₂, these results demonstrate that in finite samples, heavily weighting a subset of observations not only increases variance, but can also bias the estimate.

When the model for Q̄₀ is misspecified in simulation 2, The MLE estimator is even more biased than it was in simulation 1. The efficiency of all three double-robust efficient estimators suffers in comparison with simulation 1 as well. Nevertheless, TMLE_log, using the logistic fluctuation, has the lowest MSE of all estimators. Its superiority over TMLE_lin in terms of bias and variance is clear. TMLE_log also outperforms aug-IPTW with respect to both bias and variance, and performs much better than IPTW or MLE.

4. Discussion

When an estimation procedure incorporates weights, observations with large weights can heavily influence the point estimate and inflate the variance. Truncating these weights is a common approach to reducing the variance, but it generally introduces bias. The presented TMLE of an additive causal effect of a point treatment intervention, incorporating a logistic fluctuation of the initial conditional mean estimate, dampens the effect of these heavily weighted observations, thereby heavily reducing the reliance on truncation. As a substitution estimator, the proposed TMLE of the additive causal effect respects the global constraints of the observed data model. Simulation study results indicate that this approach is on a par with, and in the context of sparsity often superior to, fluctuating on the linear scale. In particular it is more robust when there is sparsity in the data, outperforming MLE, IPTW, and aug-IPTW.

For the sake of demonstration we considered estimation of the additive causal effect. However, the same targeted MLE, using the logistic fluctuation, can be used to estimate other point-treatment causal effects, including parameters of a marginal structural model. The newly proposed loss function also has applications in prediction of a bounded outcome, and for targeted MLE of the causal effect of a multiple time point intervention in which the final outcome is bounded and continuous. We also pointed out that the proposed fluctuation function and loss function, and corresponding targeted MLE, should also be used for continuous outcomes for which no a priori bounds are known, by simply using the minimal and maximal observed outcome values. In this way, these choices naturally robustify the targeted MLE by enforcing that the updated initial estimator will not predict outcomes outside the observed range.

The TMLE approach presented here using a logistic fluctuation of an initial estimate of the conditional mean of the continuous outcome retains all properties of targeted maximum likelihood estimators, including influence curve-based inference. The method presented here extends to collaborative targeted maximum likelihood estimation without modification.