Revisiting the g-null paradox

Abstract

The (noniterative conditional expectation) parametric g-formula is an approach to estimating causal effects of sustained treatment strategies from observational data. An often-cited limitation of the parametric g-formula is the g-null paradox: a phenomenon in which model misspecification in the parametric g-formula is guaranteed in some settings consistent with the conditions that motivate its use (i.e., when identifiability conditions hold and measured time-varying confounders are affected by past treatment). Many users of the parametric g-formula acknowledge the g-null paradox as a limitation when reporting results but still require clarity on its meaning and implications. Here we revisit the g-null paradox to clarify its role in causal inference studies. In doing so, we present analytic examples and a simulation-based illustration of the bias of parametric g-formula estimates under the conditions associated with this paradox. Our results highlight the importance of avoiding overly parsimonious models for the components of the g-formula when using this method.

Introduction

The g-formula identifies causal effects of sustained treatment strategies from observational data under no unmeasured confounding and other assumptions.¹ Unlike conventional regression, the g-formula can identify causal effects even when time-varying confounders are affected by past treatment (i.e., in the presence of so-called “treatment–confounder feedback”).^1,2 Various estimators of the g-formula have been applied in practice in numerous settings; for instance, to estimate the effect of different strategies for initiating antiretroviral therapy on the risk of death in a cohort of HIV-infected individuals.³ In this HIV example, CD4 cell count is a time-varying confounder that is affected by past treatment.

However, the g-formula is typically a high-dimensional function of measured variables when high-dimensional confounding adjustment sets are needed for identification. In this case, the g-formula may be estimated by assuming its components – the conditional outcome mean and the joint conditional density of confounders at each time given the past treatment and confounders – can be correctly characterized by parametric models. This estimator of the g-formula has been referred to as the noniterative conditional expectation parametric g-formula.^1,4–7 For simplicity, we refer to it as the parametric g-formula in this paper.

Robins and Wasserman⁸ showed that the parametric g-formula may be guaranteed some degree of model misspecification when there is treatment–confounder feedback and the sharp causal null hypothesis (i.e., the treatment has no effect on any individual’s outcome at any time) is true, even if the identifying conditions hold. As a consequence, under these conditions², a hypothesis test based on parametric g-formula estimates will falsely reject the null hypothesis of no treatment effect in large enough studies with probability approaching one.⁸ This phenomenon has been popularly referred to as the g-null paradox.

The existence of the g-null paradox is a potential threat to the validity of data analyses that rely on parametric g-formula estimates. There is, however, misunderstanding in the applied literature about the meaning and possible implications of the g-null paradox. Here we present analytic examples and a simulation-based illustration of the bias of the parametric g-formula under the conditions associated with this paradox.

The structure of the paper is as follows. We first review the observed data structure, causal estimands, and the g-formula. Then, we review the example of the g-null paradox introduced in Robins and Wasserman⁸. We clarify how model misspecification can also be guaranteed in settings other than the sharp causal null through an example. Last, we illustrate the impact of model misspecification under the conditions of the g-null paradox on bias, variance, and confidence interval coverage in a simulation study.

Background

The observed data

Consider an observational study with n individuals for which measurements are available at regularly spaced intervals (e.g., months) denoted by k = 0, …, K with k = 0 the baseline interval and K + 1 the interval in which an outcome Y is of interest. For each time k suppose the following are measured: A_k the value of a treatment of interest (e.g., dose of a given medication) and L_k a vector of covariates with L₀ possibly additionally containing time-fixed and pre-baseline covariates. We adopt the convention that L_k precedes A_k in each k and use overbars to denote the history of a random variable; e.g. ${\overline{A}}_k≔\left(A_0,A_1,\dots ,A_k\right)$.

The causal directed acyclic graph (DAG) in Figure 1a represents a possible data generating assumption for the observational study for the simple case of two times (K = 1) and with the population stratified on a single level of L₀ (such that it can be left off of the graph). Here U is an assumed unmeasured common cause of the disease outcome and a measured covariate L₁. For simplicity and without loss of generality, we assume that all covariates L_k are discrete, and that there is no missing data, no measurement error, and no death during the study period.

Figure 1:

Causal graphs for the motivating example with two time intervals. Panel A illustrates a causal DAG representing an observed data generating assumption. Panel B illustrates SWIG transformation of the causal DAG under a treatment strategy ${\overline{a}}_1$.

The causal question

Researchers are interested in using the data from this study to estimate the causal effect of an intervention that ensures everyone takes 150 mg of treatment every month during the follow-up, versus 50 mg, on the mean of the outcome. The single world intervention graph (SWIG) in Figure 1b is a transformation of the causal DAG under an intervention that sets treatment dose in the first two intervals to particular values a₀ and a₁, respectively.⁹

For a_k a possible level of treatment dose at time k and $Y^{{\overline{a}}_K}$ an individual’s outcome if, possibly contrary to fact, the individual had adhered to a strategy assigning treatment doses ${\overline{a}}_K=\left(a_0,a_1,\dots ,a_K\right)$ over the follow-up, then the average causal effect is

$$\text{E}\left(Y^{{\overline{a}}_K=\overline{150}}-Y^{{\overline{a}}_K=\overline{50}}\right).$$ (1)

Here, ${\overline{a}}_K=\overline{150}$ indicates a_k = 150 for k = 0,…, K.

The g-formula

Robins¹ showed that $\text{E}\left(Y^{{\overline{a}}_K}\right)$ can be identified by the g-formula

$$h\left({\overline{a}}_K\right)=\sum _{{\overline{l}}_K}\text{E}\left(Y|{\overline{L}}_K={\overline{l}}_K,{\overline{A}}_K={\overline{a}}_K\right)\prod _{j=0}^{K}f\left(l_j|{\overline{l}}_{j-1},{\overline{a}}_{j-1}\right),$$ (2)

where $\text{E}\left(Y|{\overline{L}}_K={\overline{l}}_K,{\overline{A}}_K={\overline{a}}_K\right)$ is the mean of Y among those with particular history $\left({\overline{l}}_K,{\overline{a}}_K\right)$ and $f\left(l_j|{\overline{l}}_{j-1},{\overline{a}}_{j-1}\right)\equiv \text{Pr}\left[L_j=l_j|{\overline{L}}_{j-1}={\overline{l}}_{j-1},{\overline{A}}_{j-1}={\overline{a}}_{j-1}\right]$ is the proportion of individuals with L_j = l_j among those with history $\left({\overline{l}}_{j-1},{\overline{a}}_{j-1}\right)$ and the sum is over all possible levels ${\overline{l}}_K$ of ${\overline{L}}_K$ in this population.

The identification of $\text{E}\left(Y^{{\overline{a}}_K}\right)$ by the g-formula (2) requires the assumptions of sequential exchangeability, positivity, and consistency which have been discussed at length elsewhere.^1,9,10 Sequential exchangeability, sometimes referred to as no unmeasured confounding, is encoded on the SWIG in Figure 1b by the absence of an arrow from the unmeasured U into the natural values of treatment at any time $\left(A_0,A_1^{a_0}\right)$.⁹ Because, for simplicity, the causal estimands here refer to so-called static strategies – strategies under which treatment assignment at each time does not depend on past patient characteristics – there will, by definition, be no arrows from measured (or unmeasured) past covariates into the intervention nodes a₀ or a₁ on the SWIG in Figure 1b as a SWIG is a transformation of the causal DAG specific to the causal estimand.⁹

The g-null paradox

The causal effect can be estimated using the n individuals in our study by estimating $h\left({\overline{a}}_K=\overline{150}\right)-h\left({\overline{a}}_K=\overline{50}\right)$. This function may be difficult to estimate in practice when (A_k, L_k) can jointly take many levels at a given time k in the data. The parametric g-formula is a computationally straightforward approach to estimating this function under the assumption that the components of the g-formula in (2) can be correctly characterized by (unsaturated) parametric models.

However, Robins and Wasserman⁸ showed that when the following three conditions hold

• Condition 1: The counterfactual mean $\text{E}\left(Y^{{\overline{a}}_K}\right)$ is identified by the g-formula $h\left({\overline{a}}_K\right)$ in (2).

• Condition 2: L_k is a time-varying confounder (in that the g-formula $h\left({\overline{a}}_K\right)$ must depend on it) and is also affected by past treatment.

• Condition 3: The treatment has no effect on any individual’s outcome at any time (i.e. the sharp null is true).

then parametric models cannot, in general, correctly characterize the g-formula (2). This contradiction, which has come to be known as the g-null paradox, implies that, if Conditions 1, 2, and 3 are true, an estimate of the effect of interest (1) using the parametric g-formula will be subject to some bias.

Robins and Wasserman⁸ illustrated this contradiction with an example, represented in Figure 2, depicting assumptions consistent with Conditions 1, 2, and 3. The example is a simplified version of our observational study with a null treatment effect, only two follow-up intervals (K = 1), constant L₀ (thus it can be ignored) and with L₁ containing only one binary covariate. In this simple case, the g-formula (2) reduces to

$$h\left(a_0,a_1\right)=\sum _{l_1=0}^1\text{E}\left(Y|l_1,a_0,a_1\right)f\left(l_1|a_0\right).$$

Figure 2:

Causal graphs of the original example by Robins and Wasserman⁸. Panel A illustrates a causal DAG representing an observed data generating assumption. Panel B illustrates a SWIG transformation of the causal DAG under a treatment strategy ${\overline{a}}_1$.

Given the joint distribution of the observed data is faithful⁸ to the graphs in Figure 2, then Figure 2 generally implies the following:

• Implication 1: $\text{E}\left(Y^{a_0,a_1}\right)=h\left(a_0,a_1\right)$ (by Condition 1) does not depend on a₀ or a₁ (by Condition 3) – consistent with the absence of any arrows into $Y^{a_0,a_1}$ except from U in Figure 2b.

• Implication 2: E(Y|l₁, a₀, a₁) does depend on l₁ (by the first part of Condition 2) – consistent with the path L₁ ← U → Y in Figure 2a.

• Implication 3: f(l₁|a₀) does depend on a₀ (by the second part of Condition 2) – consistent with the path A₀ → L₁ in Figure 2a.

Now suppose that parametric models correctly characterize the components of the g-formula:

$$\text{E}\left(Y|l_1,a_0,a_1\right)=g\left(l_1,a_0,a_1;\theta \right)$$

with g a function of (l₁, a₀, a₁) and a parameter vector θ and

$$f\left(l_1|a_0\right)=r\left(l_1,a_0;\beta \right)$$

with r a function of (l₁, a₀) that is constrained between zero and one and a parameter vector β. Given these parametric assumptions, we may replace h(a₀, a₁) with

$$h\left(a_0,a_1;\theta ,\beta \right)=\sum _{l_1=0}^{1}g\left(l_1,a_0,a_1;\theta \right)r\left(l,a_0;\beta \right).$$ (3)

Robins and Wasserman⁸ considered the following standard choices of g and r:

$$g\left(l_1,a_0,a_1;\theta \right)={\theta }_0+{\theta }_1{a}_0+{\theta }_2{a}_1+{\theta }_3{l}_1$$ (4)

$$r\left(l_1=1,a_0;\beta \right)=\frac{\text{exp}\left({\beta }_0+{\beta }_1{a}_0\right)}{1+\text{exp}\left({\beta }_0+{\beta }_1{a}_0\right)}.$$ (5)

Plugging in these specific choices of g and r into (3) we have

$$h\left(a_0,a_1;\theta ,\beta \right)={\theta }_0+{\theta }_1{a}_0+{\theta }_2{a}_1+\frac{{\theta }_3\text{exp}\left({\beta }_0+{\beta }_1{a}_0\right)}{1+\text{exp}\left({\beta }_0+{\beta }_1{a}_0\right)}.$$ (6)

For these choices of g and r, it is straightforward to see that h(a₀, a₁; θ, β) will not depend on (a₀, a₁) if and only if θ₁ = θ₂ = 0 and either θ₃ = 0 or β₁ = 0. However, θ₃ = 0 contradicts the dependence of E(Y|l₁, a₀, a₁) on l₁ and β₁ = 0 contradicts the dependence of f(l₁|a₀) on a₀.

That is, if Conditions 1, 2, and 3 hold, standard parametric models cannot correctly characterize the g-formula (2). As noted by Robins and Wasserman⁸, adding more flexibility to models g and r will not remove the problem unless these parametric models are saturated. A saturated model for g would allow this function to take a different value for each distinct combination of (l₁, a₀, a₁) and, for r, a different value for each distinct level of a₀. If treatment in our example could only take two levels in the data (e.g., if only two doses are possible in this study, 50 mg or 150 mg) then model (5) would be a saturated model for r. Adding all two- and three-way interactions to (4) would further constitute a saturated model for g in this scenario. However, even in this trivialized example with a single binary confounder L₁, saturated models for g or r are impossible/impractical to specify if treatment is continuous or otherwise can take many levels in the data. Previous authors have shown that counterexamples can be constructed for the binary treatment case where this contradiction is avoided for unsaturated models but only under scenarios where model coefficients happen to be perfect functions of others^11,12 (also see eAppendix A).

Beyond the sharp causal null hypothesis

The possibility of the g-null paradox is often handled informally in practice. Investigators dismiss the g-null paradox when they find non-null effect estimates if substantive knowledge or prior studies suggested that the sharp causal null (Condition 3) does not hold (e.g., see^13–15) or when they find null effect estimates precisely because, despite the potential for the existence of the g-null paradox, they do find a null result (e.g., see¹⁶).

This informal reasoning privileges the sharp causal null and thus obscures the broader implications of Robins and Wasserman’s example: regardless of whether the sharp causal null holds, there may be a contradiction between the assumption that parametric models can correctly characterize the g-formula and subject matter assumptions that can be encoded in a causal DAG. In other words, some model misspecification may be inevitable in realistic settings.

To see this, consider the following modification to the example in the previous section. Suppose that treatment at time 1 may affect the outcome. This scenario can be graphically represented by adding an arrow from A₁ into Y in Figure 2a and, in turn, adding an arrow from a₁ into Y in Figure 2b, further relabeling this outcome $Y^{a_1}$ to acknowledge dependence on a₁.⁹ Relying on the same general models for g and r given in (4) and (5), we again arrive at equation (6) for h(a₀, a₁; θ, β) but now consider the weaker hypothesis that this function does not depend on a₀ but allowing that it may depend on a₁. It is straightforward to see that we obtain the same contradiction: h(a₀, a₁; θ, β) will not depend on a₀ if and only if θ₁ = 0 and either θ₃ = 0 or β₁ = 0. As previously argued, the condition of θ₃ = 0 or β₁ = 0 contradicts our initial assumptions that E(Y|l₁, a₀, a₁) depends on l₁ and f(l₁|a₀) depends on a₀.

We consider this alternative scenario to illustrate the inevitability of model misspecification under the simplest possible non-null scenario. It can be seen in a similar manner that model misspecification is inevitable under other non-null scenarios. In summary, the g-null paradox is a particular instance of model misspecification that may arise when using the parametric g-formula, irrespective of whether the sharp causal null holds.

Simulations

We conducted numerical simulations to illustrate the impact of model misspecification on parametric g-formula estimates. We consider the setting with a time-varying treatment, a time-varying confounder, and an outcome measured at the end of follow-up. Data generation is loosely based on an HIV cohort study in which the treatment is an indicator of antiretroviral therapy, the confounder is an indicator of immunosuppression, and the outcome is CD4 cell count. Within this framework, data generation parameters were set to create considerable confounding and treatment–confounder feedback but no unmeasured confounding or violations of positivity. We generated data under the sharp causal null for simplicity. Similar bias may arise under non-null settings, as illustrated in the previous section.

Simulation Design

We considered six scenarios by varying the number of follow-up time points (1, 5, or 10) and the type of treatment (binary or continuous).

For each scenario, we simulated 250 longitudinal data sets with 10,000 individuals and K + 1 time points. We evaluated the bias, standard error (SE), and confidence interval (CI) coverage of the estimator for the outcome mean under each intervention and the difference of means across interventions. Standard errors were computed by taking the standard deviation of the 250 point estimates. The true outcome mean under each intervention was 500, and the true difference of means was 0.

Each data set was generated based on the DAG in Figure 2a, with the exception of including additional time points in some simulation scenarios. We first drew an unmeasured variable (U) from a Uniform(0,1) distribution. We then simulated the time-varying covariate (L_k) at each time k (k = 0,1,…, K) and simulated the outcome at time K by

$$L_k~\text{Ber}\left(p={\text{logit}}^{-1}\left({\alpha }_0+{\alpha }_1{A}_{k-1}+{\alpha }_{2}U+{\alpha }_3{A}_{k-1}U\right)\right)$$ (7)

$$Y~{\text{N}}_{\left[0,1000\right]}\left(350+300U,{50}^2\right)$$ (8)

where the value of α_i, i = 0,…,3, depended on whether the treatment was continuous or discrete and N_{[a, b]}(μ, σ²) denotes the N(μ, σ²) distribution truncated in the interval [a, b].

In the binary treatment scenarios, we set (α₀ = 0, α₁ = −2.5, α₂ = 1, α₃ = 2.5) in (7) for the simulation of L_k and simulated the time-varying treatment (A_k) at each time interval k by

$$A_k~\text{Ber}\left(p={\text{logit}}^{-1}\left(-1.25+A_{k-1}+L_k+A_{k-1}{L}_k\right)\right).$$

For the case where the treatment is continuous (e.g., treatment can be given at different doses), we set (α₀ = 1, α₁ = −0.015, α₂ = 1, α₃ = 0.015) in (7) and simulated A_k by

$$A_k~{\text{N}}_{\left[0,200\right]}\left(80+0.1A_{k-1}+30L_k-0.05A_{k-1}{L}_k,{25}^2\right).$$

We define L_k for k = −1,…,−9 as components of L₀ and generated them according to (7) with A_k = A_k−1 = 0.

Analysis of the simulated data

We considered the interventions ${\overline{a}}_K=\overline{50}$ and ${\overline{a}}_K=\overline{150}$ in the continuous treatment scenarios and considered the interventions ${\overline{a}}_K=\overline{0}$ and ${\overline{a}}_K=\overline{1}$ in the binary treatment scenarios. Of course, both would still coincide with the same interventions – but in settings where treatment is assigned differently – if the binary treatment were simply an indicator of receiving a dose of 150 (versus 50) mg. For example, the continuous treatment scenario could correspond to a setting where doctors vary widely in what dose they prescribe and the binary treatment scenario to a setting where only two doses are ever prescribed.

We applied the parametric g-formula to estimate the mean of the outcome of interest at time t = K and the difference of means under the above interventions. We computed 95% CIs around all estimates using the percentile method with 250 bootstrap replicates.

Even though Y and L_k depend on the unmeasured variable U, the variable U is not needed to obtain unbiased estimates when using the g-formula. However, Y and L_k+1 depend on the entire history of L_k through their dependence on U. Therefore, analysts need to include the entire history of L_k in their models for Y and L_k+1. The functional form of the history of L_k is unknown.

We therefore analyzed the simulated datasets with increasingly flexible models for the history of L_k.

• Least Flexible: We fit models for L_k and Y that include a single term for the (lagged) cumulative average value of L_k. In particular, we fit the following logistic model for L_k and linear model for Y

  $$\text{logit}\left(\text{Pr}\left[L_k=1|{\overline{L}}_{k-1}={\overline{l}}_{k-1},{\overline{A}}_{k-1}={\overline{a}}_{k-1}\right]\right)={\gamma }_0+{\gamma }_1{a}_{k-1}+{\gamma }_2\text{CMA}\left({\overline{l}}_{k-1}\right)$$

  $$\text{E}\left(Y|{\overline{L}}_K={\overline{l}}_K,{\overline{A}}_K={\overline{a}}_K\right)={\omega }_0+{\omega }_1{a}_K+{\omega }_2{a}_{K-1}+{\omega }_3\text{CMA}\left({\overline{a}}_{K-2}\right)+{\omega }_4\text{CMA}\left({\overline{l}}_K\right)$$

  where $\text{CMA}\left({\overline{l}}_k\right)≔\frac{1}{k+10}{\sum }_{i=-9}^k{l}_i$ denotes the cumulative average of ${\overline{l}}_k$. This analysis uses all the data required to satisfy the sequential exchangeability assumption, but it is expected to result in biased estimates because the parametric models will be somewhat misspecified.
• Moderately Flexible: We fit models for L_k and Y that include terms for the two most recent lagged values of L_k and a term for the lagged cumulative average value of L_k:

  $$\text{logit}\left(\text{Pr}\left[L_k=1|{\overline{L}}_{k-1}={\overline{l}}_{k-1},{\overline{A}}_{k-1}={\overline{a}}_{k-1}\right]\right)={\gamma }_0+{\gamma }_1{a}_{k-1}+{\gamma }_2{l}_{k-1}+{\gamma }_3{l}_{k-2}+{\gamma }_4\text{CMA}\left({\overline{l}}_{k-3}\right)$$

  $$\text{E}\left(Y|{\overline{L}}_K={\overline{l}}_K,{\overline{A}}_K={\overline{a}}_K\right)={\omega }_0+{\omega }_1{a}_K+{\omega }_2{a}_{K-1}+{\omega }_3\text{CMA}\left({\overline{a}}_{K-2}\right)+{\omega }_4{l}_K+{\omega }_5{l}_{K-1}+{\omega }_6{l}_{K-2}+{\omega }_7\text{CMA}\left({\overline{l}}_{K-3}\right).$$
• Most Flexible: We fit models for L_k and Y that include a term for each lagged value of L_k:

  $$\text{logit}\left(\text{Pr}\left[L_k=1|{\overline{L}}_{k-1}={\overline{l}}_{k-1},{\overline{A}}_{k-1}={\overline{a}}_{k-1}\right]\right)={\gamma }_0+{\gamma }_1{a}_{k-1}+\sum _{i=1}^{10}{\gamma }_{1+i}{l}_{k-i}$$

  $$\text{E}\left(Y|{\overline{L}}_K={\overline{l}}_K,{\overline{A}}_K={\overline{a}}_K\right)={\omega }_0+{\omega }_1{a}_K+{\omega }_2{a}_{K-1}+{\omega }_3\text{CMA}\left({\overline{a}}_{K-2}\right)+\sum _{i=0}^{10}{\omega }_{4+i}{l}_{K-i}.$$
• Benchmark: As a benchmark for the above three analyses, we consider an (impossible) analysis in which one has access to the unmeasured U and knowledge of the functional form of the generation for Y:

  $$\text{E}\left(Y|U=u,{\overline{A}}_K={\overline{a}}_K\right)={\omega }_0+{\omega }_1{a}_K+{\omega }_2{a}_{K-1}+{\omega }_3\text{CMA}\left({\overline{a}}_{K-2}\right)+{\omega }_{4}u.$$

  Because U is a baseline covariate, the distribution of U can be estimated nonparametrically by the empirical distribution of U. This analysis will be unbiased.

One could similarly explore the impact of increasing the flexibility of the history of treatment in the models for L_k and Y. While we did not include such analyses to simplify the presentation, a reduction of bias would require an extremely flexible function of ${\overline{A}}_{k-1}$ in the L_k model because the true relation between ${\overline{A}}_{k-1}$ and L_k is highly complex, as the result of our choice to include an interaction term between A_k−1 and the unmeasured variable U in the data generating model for L_k (also see eAppendix B). We applied the parametric g-formula using the gfoRmula R package⁶. The code used for all analyses is available on GitHub at https://github.com/CausalInference/NullParadox.

Results

Figure 3 illustrates the simulation results for the mean difference. The bias, standard error (SE), and confidence interval (CI) coverage of the parametric g-formula are summarized in Table 1.

Figure 3:

Violin plots illustrating the estimated average causal effect of the four applications of the parametric g-formula in the binary treatment scenarios (panel A) and continuous treatment scenarios (panel B). Darker shading indicates simulation settings with larger number of time points (light shading: K = 1; medium shading: K = 5; dark shading: K = 10). The dashed line indicates the true value of the average causal effect (i.e., 0). ACE = average causal effect.

Table 1:

Simulation results for the mean difference. The target parameter in the binary treatment scenarios is $\text{E}\left(Y^{{\overline{a}}_K=\overline{1}}\right)-\text{E}\left(Y^{{\overline{a}}_K=\overline{0}}\right)$ and the target parameter in the continuous treatment scenarios is $\text{E}\left(Y^{{\overline{a}}_K=\overline{150}}\right)-\text{E}\left(Y^{{\overline{a}}_K=\overline{50}}\right)$. The true value of the target parameters is 0. SE = standard error.

		Binary Treatment Scenarios			Continuous Treatment Scenarios
G-Formula Application	K	Bias	SE	Coverage	Bias	SE	Coverage
Least Flexible	1	4.26	2.39	0.54	18.19	4.54	0.01
	5	18.17	3.44	0.00	48.73	8.04	0.00
	10	21.14	4.74	0.00	50.39	9.77	0.00
Moderately Flexible	1	1.42	2.46	0.86	0.57	4.76	0.98
	5	10.50	3.52	0.20	19.63	8.19	0.33
	10	15.07	4.88	0.12	34.43	10.01	0.07
Most Flexible	1	1.42	2.46	0.86	0.55	4.76	0.98
	5	4.65	3.65	0.82	−5.49	8.52	0.91
	10	−1.05	5.13	0.94	−13.53	10.38	0.73
Benchmark	1	0.11	1.22	0.96	0.10	2.77	0.91
	5	0.05	2.10	0.93	−0.09	4.52	0.94
	10	0.03	2.81	0.93	0.10	6.24	0.94

The performance of the parametric g-formula generally improved as the flexibility of the models for L_k and Y increased. That is, the impact of model misspecification was greatly mitigated, but not completely eliminated, by using more flexible models for the components of the parametric g-formula. For instance, at K = 10 in the continuous treatment scenario, the least flexible application of the parametric g-formula had a bias of 50.39, SE of 9.77, and coverage of 0.00 whereas the most flexible application had a bias of −13.53, SE of 10.38, and coverage of 0.73.

The simulation results for the counterfactual means are given in eAppendix B. We observed the same trends: the performance of the parametric g-formula improved when using more flexible models. Moreover, we found similar results when increasing the number of iterations in the simulations.

Discussion

Our presentation clarifies that the g-null paradox of the noniterative conditional expectation parametric g-formula is a particular case of a general form of model misspecification that may occur even if the treatment of interest has a non-null causal effect. Part of the confusion surrounding the g-null paradox arose because the paradox has been traditionally discussed in the context of testing the sharp causal null hypothesis of no treatment effect. For example, Campbell and Gustafson¹⁷ investigated the empirical type I error rate based on the parametric g-formula. They did not find higher empirical type I error rates under model misspecification compared with saturated models without model misspecification, although their sample sizes were smaller than those considered here.

However, because the primary goal of causal inference is estimating treatment effects rather than “testing hypotheses”, researchers are often concerned about the magnitude of bias in their estimates. Thus, one may view the g-null paradox as simply a phenomenon in which the parametric g-formula estimate of a treatment effect is guaranteed to be biased due to model misspecification, regardless of whether or not the null is true. Although not the focus of their simulation study, Murray et al.¹⁸ found the bias of the parametric g-formula to be negligible in scenarios of a null treatment effect. In contrast, our more extensive simulations illustrate that a nonnegligible amount of bias can arise in some simple scenarios even when using fairly flexible models for the components of the parametric g-formula.

Evaluating model misspecification in the parametric g-formula may be informally done by conducting sensitivity analyses under different modeling assumptions and under different orderings of the factorization of the joint density of the confounders (particularly in interval cohorts), comparing the parametric g-formula and the non-parametric estimate of the outcome mean/risk mean under the natural course.¹⁹ Additionally, data analysts may consider applying approaches that are based on different algebraic representations of the g-formula, such as inverse probability (IP) weighted estimators or the iterative conditional expectation (ICE) parametric g-formula, which rely on different modeling assumptions. State-of-the-art^20–24 methods derived from the so-called efficient influence function are increasingly available.^25–27

Because machine learning (ML) methods relax parametric modeling assumptions, they may naturally be considered a possible solution to model misspecification in the parametric g-formula. Yet, perhaps counterintuitively, the problem of model misspecification in the parametric g-formula is not reasonably solved by using ML algorithms to estimate its nuisance functions (i.e., the joint conditional density of the covariates and conditional outcome mean). Recent simulation studies clarified that, while so-called state-of-the-art methods can benefit from use of ML algorithms^24,28, the ML-based singly-robust estimators (i.e., estimators, such as the parametric g-formula, that require all models for the nuisance functions to be correctly specified in order for the estimator to be consistent) do not enjoy this benefit and may perform worse than those based on parametric models.²⁹ Therefore, we did not include such approaches in our simulations.

In summary, model misspecification may introduce bias in parametric g-formula estimates, regardless of whether treatment has a null causal effect. Thus, overly parsimonious models for the components of the g-formula are to be avoided.

Data / code availability:

The code used for all analyses is available on GitHub at https://github.com/CausalInference/NullParadox.