Causal Estimation with Functional Confounders

Abstract

Causal inference relies on two fundamental assumptions: ignorability and positivity. We study causal inference when the true confounder value can be expressed as a function of the observed data; we call this setting estimation with functional confounders (EFC). In this setting ignorability is satisfied, however positivity is violated, and causal inference is impossible in general. We consider two scenarios where causal effects are estimable. First, we discuss interventions on a part of the treatment called functional interventions and a sufficient condition for effect estimation of these interventions called functional positivity. Second, we develop conditions for nonparametric effect estimation based on the gradient fields of the functional confounder and the true outcome function. To estimate effects under these conditions, we develop Level-set Orthogonal Descent Estimation (LODE). Further, we prove error bounds on LODE’s effect estimates, evaluate our methods on simulated and real data, and empirically demonstrate the value of EFC.

1. Introduction

Determining the effect of interventions on outcomes using observational data lies at the core of many fields like medicine, economic policy, and genomics. For example, policy makers estimate effects to elect whether to invest in education or job training programs. In medicine, doctors use effects to design optimal treatment strategies for patients. Geneticists perform genome-wide association studies (GWAS) to relate genotypes and phenotypes. In observational data, there could exist unobserved variables that affect both the intervention and the outcome, called confounders. A necessary condition for the causal effect to be identified is that all confounders are observed; called ignorability. If ignorability holds, a sufficient condition for causal effect estimation is adequate variation in the intervention after conditioning on the confounders; this condition is called positivity.

The data apriori does not differentiate between confounders and interventions. It is the practitioners that select interventions of interest from all pre-outcome variables (variables that occur before the outcome). Then, assuming knowledge of the data generating mechanism, practitioners can label certain variables amongst the remaining pre-outcome variables as confounders. This corresponds to indexing into the set of pre-outcome variables.

In certain problems the confounders are specified as a function of the pre-outcome variables that does not simply index into the set of pre-outcome variables. For a concrete example, consider GWAS. The goal in GWAS is to estimate the influence of genetic variations on phenotypes like disease risk. In GWAS, population and family structures both result in certain genetic variations and affect phenotypes and therefore, are confounders [4]. Practitioners specify these confounders by using the genetic similarity between individuals [15, 19, 31], which is a function of the genetic variations. When the confounders are a function of the same pre-outcome variables that define the interventions, positivity is violated. Then, the class of interventions whose effects are estimable is not well-defined.

We study causal effect estimation in such settings, where a function of the pre-outcome variables provides the confounder and these same pre-outcome variables define the intervention. We call this estimation with functional confounders (EFC). In EFC, one column in the observed data is the outcome and all others are pre-outcome variables. We assume access to a function h(·) that takes as input the pre-outcome variables and returns the value of the confounder. Further, we assume these confounders give us ignorability. In settings like GWAS, the function h reflects the practitioner-specified function that captures the genetic variation influenced by the population structure. In traditional observational causal inference (OBS-CI), h(·) reflects the selection of certain variables in the data and labelling them as confounders. In EFC, two different values of the confounder are never observed for the same setting of the pre-outcome variables. This means that positivity is violated and the effects of only certain interventions may be estimable.

We address this issue in two ways. First, we investigate a class of plausible interventions that are functions of the observed pre-outcome variables, called functional interventions. We develop a sufficient condition to estimate the effects of said functional interventions, called functional positivity (F-POSITIVITY). Second, we consider intervening on all pre-outcome variables, called the full intervention. We develop a sufficient condition to estimate the effect of the full intervention, called causal redundancy (C-REDUNDANCY). For an intervention, given a confounder value, C-REDUNDANCY allows us to compute a surrogate intervention such that the conditional effect of the surrogate is equal to that of the original intervention. We also show that such surrogate interventions exist only under a certain condition that we call Effect Connectivity, that is necessary for nonparametric effect estimation in EFC. This condition is satisfied by default in traditional OBS-CI if ignorability and positivity hold. Then, we develop an algorithm for causal estimation assuming C-REDUNDANCY, called Level-set Orthogonal Descent Estimation (LODE), which estimates effects using surrogate interventions. If the surrogate is not estimated well, LODE’s estimates are biased. We establish bounds on this bias that capture the mitigating effect of the smoothness of the true outcome function.

Related work

The problem of genome-wide association studies (GWAS) is to estimate the effect of genetic variations(also called single nucleotide polymorphisms (SNPs)) on the phenotype [29]. The ancestry of the subjects acts as a confounder in GWAS. In GWAS practice, principle component analysis (PCA) and linear mixed models (LMMs) are used to compute this confounding structure [19, 31]. Lippert et al. [15] suggest estimating the confounders and effects on separate subsets of the SNPs. This separation disregards the confounding that is captured in the interaction of the two subsets of SNPs. GWAS is a special case of effects from multiple treatments (MTE) where the confounder value is specified via optimization as a function of the pre-outcome variables [20, 30]. In all these settings, positivity is violated and not all effects are estimable. We provide an avenue for nonparametric effect-estimation of the full intervention under a new condition, C-REDUNDANCY.

Traditional observational causal inference (OBS-CI) review

We setup causal inference with Structural Causal Models [17] and use do(t = t*) to denote making an intervention. Let t be a vector of the interventions, z be the confounder, and y be the outcome. Let η ~ p(η)(η⫫(z, t)) be noise. With f as the outcome function, we define the causal model for traditional OBS-CI as¹:

$$z~\text{p}(z),t~\text{p}(t|z),\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}y}=\text{f}(t,z,\eta ).$$

Let p(y, z, t) denote the joint distribution implied by this data generating process. The effects of interest under the full intervention do(t = t*) are the average and conditional effect

$$\text{(average)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}\tau \left({\text{t}}^{*}\right)={\mathbb{E}}_{z,\eta }\text{f}\left({\text{t}}^{*},z,\mathbf{\eta }\right)\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}(conditional)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}\varphi \left({\text{t}}^{*},\text{z}\right)={\mathbb{E}}_{\eta }\left[\text{f}\left({\text{t}}^{*},\text{z},\eta \right)\right].$$ (1)

With observed confounders, two assumptions make causal estimation possible: ignorability and positivity. Ignorability means that all confounders z are observed in data. Conditioning on all the confounders, the outcome under an intervention is distributed as if conditional on the value of the intervention: p(y = y₁ | do(t = t*),z = z) = p(f(t*, z, η) = y₁) = p(y = y₁ | t = t*,z = z). This allows the expression of average effect as an expectation over the observed outcomes $\tau \left({\text{t}}^{*}\right)={\mathbb{E}}_{z,\eta }\left[\text{f}\left({\text{t}}^{*},z,\eta \right)\right]={\mathbb{E}}_z\mathbb{E}\left[\text{y}|z,{\text{t}}^{*}\right]$. The conditional expectation only exists for all t* if p(y | z, t = t*) = ^p(y,z,t=t*)/_{p(z)p(t=t* | z}) exists. Positivity guarantees this existence

$$\text{(positivity)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}}\forall {\text{t}}^{*}\in \text{supp}(t)\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}p}(z=\text{z})>0\Rightarrow \text{p}\left(t={\text{t}}^{*}|z=\text{z}\right)>0.$$ (2)

2. Estimation with functional confounders

In traditional OBS-CI, causal estimation relied on knowing the confounders. In this section, we consider settings where confounders are known via a function of the pre-outcome variables h(t) = z. We call this setting estimation with functional confounders (EFC). An example of this is GWAS, where SNPs (the pre-outcome variables) are used to estimate the confounding population structure through methods like PCA [31]. Assuming the confounders are a function of the pre-outcome variables violates positivity in general. Positivity is violated in this setting because

$$\forall {\text{t}}_1,{\text{t}}_2\in \text{supp}(t)\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}s.t.\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}h}\left({\text{t}}_2\right)\ne \text{h}\left({\text{t}}_1\right)\Rightarrow \text{p}\left(z=\text{h}\left({\text{t}}_2\right)|t={\text{t}}_1\right)=0\ne \text{p}\left(z=\text{h}\left({\text{t}}_2\right)\right)>0$$

In words, two different confounder values cannot occur for the same t. A positivity violation precludes nonparametric effect estimation of the full intervention do(t = t*).

Positivity and Regression Identifiability

Positivity can be viewed as providing identifiability. To see this, let the confounder be z = h(t) and the outcome be y(t, z, η) = z + h(t). Now consider regressing z and t onto y. Then, functions y = αz + βh(t) indexed by α, β, such that α + β = 2, are consistent with the observed data. Thus, there exist infinitely many solutions to the conditional expectation of y on (t, z), meaning that the regression is not identifiable. Assuming positivity necessitates sufficient randomness to identify the regression and thus the causal effect. A violation of positivity means that nonparametric estimation of causal effects needs further assumptions.

2.1. Setup for EFC

In EFC, the confounder is provided as a non-bijective function h of the pre-outcome variables t. To reflect this property, we use h(t) to denote the confounder. As an illustrative example, let [mi] be the Gamma distribution and consider z ∈ {−1, 1}, p(z = 1) = 0.5 is the confounder and the intervention of interest is $t=z*\mathcal{G}(1,\text{\hspace{0.17em}exp}(z))$. Note sign(t) = z meaning that h(t) = sign(t) is the confounder. Figure 1 shows causal graphs connecting our EFC notation to that in traditional OBS-CI. With noise η ~ p(η)(η⫫t), our causal model samples, in order, the confounder ”part” of pre-outcome variables h(t), the pre-outcome variables t, and the outcome y via the outcome function f²:

$$\text{h}(t)~\text{p}(\text{h}(t))t~\text{p}(t|\text{h}(\text{t}))y=\text{f}(t,\text{h}(t),\eta )$$

Figure 1:

Causal Graphs for Traditional OBS-CI vs. EFC.

Similar to traditional OBS-CI, for an intervention t* the average effect, τ(·), and the conditional effect, ϕ(·,·) at $\text{h}\left({\text{t}}_2^{*}\right)$, respectively, are defined as:

$$\tau \left({\text{t}}^{*}\right)=\underset{\text{h}(t),\eta }{\mathbb{E}}\left[\text{f}\left({\text{t}}^{*},\text{h}(t),\eta \right)\right],\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\underset{\eta }{\mathbb{E}}\left[\text{f}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right),\eta \right)\right].$$ (3)

As the pre-outcome variables determine the confounder, positivity is violated. Further, the outcome function f(t, h(t), η) could recover the exact value of h(t) from t instead of its second argument. Thus, two different outcome functions could lead to the same observational data distribution, posing a fundamental obstacle to causal effect estimation. This is the central challenge in EFC.

2.2. Causal Questions With Functional Positivity

Without positivity, we can only estimate the effects of certain functions of t. We call such interventions, on some function g(t), functional interventions. The implied causal model for the outcome for functional intervention value g(t*) and confounder value $\text{h}\left({\text{t}}_2^{*}\right)$ is first t ~ p(t | g(t) = g(t*), $t~\text{p}\left(t|\text{g}(t)=\text{g}\left({\text{t}}^{*}\right),\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)$ and then $y=\text{f}\left(t,\text{h}\left({\text{t}}_2^{*}\right),\eta \right)$³. Then, the functional average effect is

$$(\text{average\hspace{0.17em}})\tau \left(\text{g}\left({\text{t}}^{*}\right)\right)={\mathbb{E}}_{\text{h}(t),\eta }{\mathbb{E}}_{t|\text{g}(t)=\text{g}\left({\text{t}}^{*}\right),\text{h}(t)}[\text{f}(t,\text{h}(t),\eta )]$$

An example of a functional intervention is intervening on the cumulative dosage of a drug. In contrast, traditional interventions would set each individual dose given at different points in time.

F-POSITIVITY and Functional Effect Estimation

For the causal model above to be well-defined for all functional interventions g(t*), the conditional $\text{p}\left(t|\text{g}(t)=\text{g}\left({\text{t}}^{*}\right),\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)$ must exist. To guarantee this existence, we define functional positivity (F-POSITIVITY) for any g(t*)

$$\text{(F-POSITIVITY)\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}p}\left(\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0\Rightarrow \text{p}\left(\text{g}(t)=\text{g}\left({\text{t}}^{*}\right)|\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0.$$ (4)

F-POSITIVITY says that the function of the pre-outcome variables that is being intervened on needs to have sufficient randomness when the function of the pre-outcome variables that defines the confounders is fixed. Further, under F-POSITIVITY, effect estimation for functional interventions is reduced to traditional OBS-CI on data p(y, g(t), h(t)). With positivity and ignorability satisfied, traditional causal estimators such as propensity scores [23], matching [21], regression [11], and doubly robust methods [22] can be used to estimate the causal effect. Focusing on regression, let f_θ be a flexible function, then ${\text{min}}_{\theta }{\mathbb{E}}_{\text{y},t}\left[{\left(y-{\text{f}}_{\theta }(\text{h}(t),\text{g}(t))\right)}^2\right]$ would estimate the conditional expectation of interest : $\mathbb{E}\left[y|\text{h}(t),\text{g}\left({\text{t}}^{*}\right)\right]$. With θ, the effect of g(t*) can be estimated by averaging the estimate of the conditional expectation over the marginal distribution p(h(t)):

$$\tau \left(\text{g}\left({\text{t}}^{*}\right)\right)={\mathbb{E}}_t\left[{\text{f}}_{\theta }\left(\text{h}(t),\text{g}\left({\text{t}}^{*}\right)\right)\right].$$ (5)

3. Identification of effects of the full intervention

When positivity is violated, causal effects cannot be estimated as conditional expectations over the observed data in general. We give a functional condition, called causal redundancy (C-REDUNDANCY), that allows us to estimate the effect of the full intervention do(t = t*), even when positivity is violated. Specifically, C-REDUNDANCY allows us to construct a surrogate intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ whose conditional effect at h(t′) matches the conditional effect of interest, $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$. Let $\tilde{\text{t}}$ be a fixed value of the full intervention, then C-REDUNDANCY is

Assumption. Recall the outcome $\text{y}=\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$. With ${\nabla }_{\tilde{\text{t}}}$ as gradient w.r.t. to argument $\tilde{\text{t}}$:

$$\forall \tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta ,{\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0.$$

In words, C-REDUNDANCY is the condition that the outcome function f uses the value of the confounder from its second argument instead of computing h(t) from the first argument⁴. To compute the conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, we develop Level-set Orthogonal Descent Estimation (LODE). LODE’s key step is to construct a surrogate intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ such that

$$\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right),\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}h}\left({\text{t}}_2^{*}\right)=\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right).$$

By definition, a surrogate intervention lives in the conditional effect level-set: $\left\{\tilde{\text{t}}:\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right\}$. So LODE searches this level-set for ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$. See fig. 2 which plots the conditional effect level-sets with the value of h(t) fixed (red) in (supp(t), supp(h(t)))-space. Green corresponds to the observed data, supp(t, h(t)). LODE finds ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ by traversing the level-sets (black) to account for the confounder part mismatch $\text{h}\left({\text{t}}^{*}\right)\ne \text{h}\left({\text{t}}_2^{*}\right)$. C-REDUNDANCY ensures LODE can traverse these level-sets as it implies ${\nabla }_{\tilde{\text{t}}}\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0$ under the regularity conditions in theorem 1. Thus, under C-REDUNDANCY, surrogate interventions can be constructed by solving a gradient flow equation which guarantees identification as follows:

Figure 2:

LODE’s traversal.

Theorem 1. Assume C-REDUNDANCY holds. Assuming the following:

1. Let ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ be the limiting solution to the gradient flow equation $\frac{\text{d}\tilde{\text{t}}(\text{s})}{\text{ds}}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, initialized at $\tilde{\text{t}}(0)={\text{t}}^{*}$; i.e. ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)={\text{lim}}_{\text{s}\to \infty }\tilde{\text{t}}(\text{s})$.

   Further, let $\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)=\text{h}\left({\text{t}}_2^{*}\right)$ and ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\in \mathit{\text{supp}}(t)$.

2. $\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$ and $\text{h}(\tilde{\text{t}})$ as functions of $\tilde{\text{t}}$, $\text{h}(\tilde{\text{t}})$ are continuous and differentiable and the derivatives exist for all $\tilde{\text{t}}$, η. Let ${\nabla }_{\tilde{\text{t}}}\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$ exist and be bounded and integrable w.r.t. the probability measure corresponding to p(η), for all values of $\tilde{\text{t}}$ and $\text{h}(\tilde{\text{t}})$.

Then the conditional effect (and therefore the average effect) is identified:

$$\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)\right)=\mathbb{E}\left[y|t={\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right]$$ (6)

In words, the key idea is that starting at $\tilde{\text{t}}(0)={\text{t}}^{*}$ and following ${\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})$ means $\tilde{\text{t}}(\text{s})$ always lies in the level-set $\left\{\tilde{\text{t}}:\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right\}$. See appendix A.2 for the proof. While C-REDUNDANCY is stated in terms of the gradient of the outcome function, it suffices for theorem 1 to assume a weaker condition about the gradient of the conditional effect: ${\nabla }_{\tilde{\text{t}}}{\mathbb{E}}_{\eta }\text{f}{\left(\tilde{\text{t}},{\tilde{\text{t}}}_2,\mathbf{\eta }\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0$.

Surrogate Positivity

In theorem 1, we assumed that the surrogate ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ ∈ supp(t). This condition, which we call surrogate positivity (analogous to positivity), states that for any intervention and confounder, surrogate interventions that are limiting solutions to the gradient flow equation have nonzero density conditional on the confounder value. Formally, for any intervention t = t*

$$\text{p}\left(\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0\Rightarrow \text{p}\left(t={\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0,$$ (7)

and ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ satisfies assumption 1 in theorem 1. Surrogate positivity along with C-REDUNDANCY, is sufficient for full effect estimation under EFC. Next, we show that the positivity assumption in traditional causal inference is a special case of surrogate positivity.

Traditional observational causal inference (OBS-CI) and LODE

Let the confounder and intervention of interest in traditional OBS-CI be z and a respectively. Assume both are scalars and ignorability and positivity hold. This setup can be embedded in EFC by defining the vector of pre-outcome variables as: t = [a; z]. In this setting, C-REDUNDANCY and surrogate positivity(eq. (7)) hold by default. Let the outcome be y = f(t, h(t)) = f(a, z), where f only depends on the first element of t, i.e. a⁵. Let e₁ = [1, 0] and e₂ = [0,1]. In traditional OBS-CI as EFC, ${\nabla }_{\tilde{\text{t}}}\text{f}\left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)\propto {\text{e}}_1$ and ${\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})\propto {\text{e}}_2$ meaning that ${\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0$. Thus, C-REDUNDANCY holds by default. Moreover, under positivity of a w.r.t. z, we also have surrogate positivity for traditional OBS-CI as an EFC problem. In this setting, LODE computes ${\text{t}}^{\prime }=\left[{\text{a}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right]$ by following $-{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=[0,-1]$, which only changes the value of $\text{h}\left({\tilde{\text{t}}}_2\right)$, not the value of a. Thus, t* and ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ will have the same first element and t′’s second element will be $\text{h}\left({\text{t}}_2^{*}\right)$. As a has positivity w.r.t. z, we have $\text{p}\left(a={\text{a}}^{*},z=\text{h}\left({\text{t}}_2^{*}\right)\right)>0$ which means t′ ∈ supp(t). The estimated conditional effect is $\mathbb{E}\left[y|t={\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right]=\text{f}\left(\left[{\text{a}}^{*},z^{*}\right],\text{h}\left({\text{t}}_2^{*}\right)\right)=\mathbb{E}\left[y|a={\text{a}}^{*},z=\text{h}\left({\text{t}}_2^{*}\right)\right)]$ which matches the estimate in traditional OBS-CI.

Implementation of LODE

LODE first estimates the conditional expectation $\mathbb{E}[y|t]$; this can be done with model-based or nonparametric estimators. This is achieved by regressing y on t, $\widehat{\text{f}}={\text{arg\hspace{0.17em}min}}_{\text{u}\in \mathcal{F}}{\mathbb{E}}_{y,t~\text{D}}{(y-\text{u}(t))}^2$, with empirical distribution D. The surrogate intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ is computed using Euler integration to solve the gradient flow equation. Euler integration in this setting is equivalent to gradient descent with a fixed step size. Other, more efficient schemes like Runge–Kutta numerical integration methods [3] could also be used. The conditional effect estimate is $\widehat{\text{f}}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)$. See algorithm 1 for a description.

3.1. Estimation error of LODE in practice

To compute the surrogate intervention t′, LODE uses the gradients of h(·) in Euler integration. In practice, taking Euler integration steps, instead of solving the gradient flow exactly, could result in errors. Then t′ could lie outside the level-set of the conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)={\mathbb{E}}_{\eta }\left[\text{f}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right),\eta \right)\right]$. Further, if $\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)\ne \text{h}\left({\text{t}}_2^{*}\right)$, LODE incurs error for conditioning on a value of the confounder that is different from $\text{h}\left({\text{t}}_2^{*}\right)$. The error due to t′ estimation is decoupled from the error in the estimation of $\mathbb{E}[y|t]$ which adds without further amplification. We formalize this error:

Theorem 2. Consider the conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$. Let $\widehat{\text{t}}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ be the estimate of the surrogate intervention computed by LODE, computed via Euler integration of the gradient flow $\frac{\text{d}\tilde{\text{t}}(\text{s})}{\text{ds}}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, initialized at $\tilde{\text{t}}(0)={\text{t}}^{*}$. Assume the true surrogate ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ exists and is the limiting solution to the gradient flow equation.

1. Let the finite sample estimator of $\mathbb{E}[y|t=\tilde{\text{t}}]$ be $\widehat{\text{f}}(\tilde{\text{t}})$. Let the error for all $\tilde{\text{t}}$ be bounded, $|\widehat{\text{f}}(\tilde{\text{t}})-\mathbb{E}[y|t=\tilde{\text{t}}]|⩽\text{c}\left(\text{N}\right)$, where N is the sample size and lim_N→∞ c(N) = 0.

2. Assume K Euler integrator steps were taken to find the surrogate estimate $\widehat{\text{t}}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, each of size ℓ. Let the maximum confounder mismatch be ${\text{max}}_{\text{i}⩽\text{K}}{\left(\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2=\text{M}$.

3. Let ${\text{L}}_{z,\tilde{\text{t}}}$ be the Lipschitz-constant of $\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)$ as a function of $\text{h}\left({\tilde{\text{t}}}_2\right)$, for fixed $\tilde{\text{t}}$.

   Let L_e be the Lipschitz-constant of $\mathbb{E}[y|t=\tilde{\text{t}}]=\varphi (\tilde{\text{t}},\text{h}(\tilde{\text{t}}))$ as a function of $\tilde{\text{t}}$.

   Assume h has a gradient with bounded norm, ${\Vert \nabla \text{h}(\tilde{\text{t}})\Vert }_2<{\text{L}}_{\text{h}}$.

   Assume f’s Hessian has bounded eigenvalues: $\forall \tilde{\text{t}},{\tilde{\text{t}}}_2$, ${\Vert {\nabla }_{\tilde{\text{t}}}^2\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)\Vert }_2⩽{\sigma }_{\text{H}\varphi }$.

The conditional effect estimate error, $\xi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=|\widehat{\text{f}}(\widehat{\text{t}})-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|$, is upper bounded by:

$$\text{c}(\text{N})+\text{min\hspace{0.17em}}\left({\text{L}}_{\text{e}}{\Vert {\text{t}}^{\prime }-\widehat{\text{t}}\Vert }_2,2\text{K}{\ell }^2\left(\mathcal{O}(\ell )+\text{M}{\sigma }_{\text{H}\varphi }{\text{L}}_{\text{h}}^2\right)+{\text{L}}_{z,\widehat{\text{t}}}{\Vert \text{h}(\widehat{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\Vert }_2\right)$$ (8)

See appendix A.3 for the proof. Theorem 2 captures the trade-off between biases due to conditioning on the wrong confounder value and due to the accumulated error in solving the gradient flow equation. This accumulated error analysis may be loose in settings where the sum of many gradient steps lead to $\widehat{\text{t}}\approx {\text{t}}^{\prime }$, even if each step individually induces large error. In such settings, the term that depends on ${\Vert \widehat{\text{t}}-{\text{t}}^{\prime }\Vert }_2$ is a better measure of error. The maximum-mismatch [mi] appears because Euler integrator takes steps that depend on the magnitude of the gradient which depends on the mismatch value $\left(\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)$. If mismatch is large for some i, the Euler step could lead to a large error for a fixed step size ℓ. We discuss the assumptions in theorems 1 and 2 in appendix A.1

3.2. Effect Connectivity and the Existence of ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$

The key element in Theorem 1 is the surrogate intervention t′ such that its conditional effect given h(t′), equals that of t* and $\text{h}\left({\text{t}}_2^{*}\right)$. The orthogonality ${\nabla }_{\tilde{\text{t}}}{\text{f}}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}=0$, is a functional condition that does not guarantee ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ exists in supp(t); a necessity to compute $\mathbb{E}\left[y|t={\text{t}}^{\prime }\right]$ without additional parametric assumptions. We give a general condition called Effect Connectivity that guarantees the surrogate intervention exists. With conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, for any t*

$$\text{p}\left(\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0\Rightarrow \text{p}\left(\varphi (t,\text{h}(t))=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0.$$ (9)

In words, t has a chance of setting the conditional effect to any possible value supp(ϕ(t, h(t₂))) given any confounder value $\text{h}\left({\text{t}}_2^{*}\right)\in \text{supp}(\text{h}(\text{t}))$. An equivalent statement is that every level set of the conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, with $\text{h}\left({\text{t}}_2^{*}\right)$ fixed, contains an intervention for each confounder value. That is, for some $\text{h}\left({\text{t}}_2^{*}\right)$ define the level set ${\text{A}}_{\text{c}}=\left\{{\text{t}}^{*};\text{f}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\text{c}\right\}$, then $\forall \text{h}\left({\text{t}}_2^{*}\right)\in \text{supp}(\text{h}(t))$, $\text{p}\left(t\in {\text{A}}_{\text{c}}|\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0$.

Theorem 3. Under Effect Connectivity, eq. (9), any surrogate intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\in \mathit{\text{supp}}(t)$.

We give the proof in appendix A.4. Whether the intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ can be found via tractable search is problem-specific. If the surrogate ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ exists $\forall {\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)$, then eq. (9) holds by definition of the surrogate. Effect Connectivity allows us to reason about values of f anywhere in supp(t) × supp(h(t)) using only samples from p(y, t). Further, it is necessary in EFC:

Theorem 4. Effect Connectivity is necessary for nonparametric effect estimation in EFC.

We prove this in appendix A.5. Effect Connectivity ensures that causal models with different causal effects have different observational distributions. Then, parametric assumptions on the causal model are not necessary to estimate effects.

4. Experiments

We evaluate LODE on simulated data first and show that LODE can correct for confounding. We also investigate the error induced by imperfect estimation of the surrogate intervention in LODE. Further, we run LODE on a GWAS dataset [6] and demonstrate that LODE is able to correct for confounding and recovers genetic variations that have been reported relevant to Celiac disease [8, 25, 14, 1].

4.1. Simulated experiments

We investigate different properties of LODE on simulated data where ground truth is available. Let the dimension of t (pre-outcome variables) be T = 20 and outcome noise be $\eta ~\mathcal{N}(0,0.1)$. We consider two EFC causal models, denoted by A and B with different h(t) and f(t, h(t), η):

1. $$\text{h}(t)=\gamma \frac{{\sum }_{\text{i}}{t}_{\text{i}}}{\sqrt{\text{T}}},t~\mathcal{N}\left(0,{\sigma }^2{\mathbb{I}}^{\text{T}\times \text{T}}\right),\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}y}=\frac{{\sum }_{\text{i}}{\left(-1\right)}^{\text{i}}{t}_{\text{i}}}{\sqrt{\text{T}}}+\alpha \text{h}{(t)}^2+(1+\alpha )\text{h}(t)+\eta$$
2. $$\text{h}(t)=\sum _{\text{i}:\text{i}\in 2\mathbb{Z}}\gamma t_{\text{i}}{t}_{\text{i}+1},t~\mathcal{N}\left(0,{\sigma }^2{\mathbb{I}}^{\top \times \top }\right),\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}y}=\frac{{\sum }_{\text{i}}{(-1)}^{\text{i}}{t}_{\text{i}}^2}{\sqrt{\text{T}}}+\alpha \text{h}(t)+\eta$$

In both causal models, C-REDUNDANCY is satisfied. The constant γ controls the strength of the confounder and the constant α controls the Lipschitz constant of the outcome as a function of the confounder. We let the variance σ² = 1, unless specified otherwise. In the following, we train on 1000 samples and report conditional effect root-mean-squared error (RMSE), computed with another 1000 samples. We used a degree-2 kernel ridge regression to fit the outcome model as a function of t. This model is correctly specified, and so the conditional $\mathbb{E}[y|t=\tilde{\text{t}}]$ can be estimated well. We compare against a baseline estimate of conditional effect that is the same outcome model’s estimate of $\mathbb{E}\left[y|t={\text{t}}^{*}\right]$. This baseline fails to account for confounding and produces a biased estimate of the conditional effect of do(t = t*), conditional on any $\text{h}\left({\text{t}}_2^{*}\right)\ne \text{h}\left({\text{t}}^{*}\right)$.

First, we investigate how well LODE can correct for confounding for both causal models. We let α = 1 and obtain surrogate estimates by Euler integrating until the quantity ${\mathbb{E}}_{{\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)}{\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$ is smaller than 10⁻⁴ times value at initialization, where ${\mathbb{E}}_{{\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)}$ is expectation over the evaluation set. In fig. 3, we plot the mean and standard deviation of conditional effect RMSE averaged over 10 seeds, for different strengths of confounding. We see that LODE is able to estimate effects well across multiple strengths of confounding while the baseline suffers.

Figure 3:

RMSE of estimated conditional effect vs. strength of confounding γ. LODE corrects for confounding and produces good effect estimates across different values of γ.

Second, we investigate LODE’s estimation when surrogate positivity holds but the probability $\text{p}\left(t\approx {\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)$ is very small. This results in estimation error due to poor fitting of the outcome model in low density regions of supp(t). We run LODE on simulated data where t is generated with different variances (σ²). For small σ, the outcome model error is large when using surrogate interventions ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, where either $\text{h}\left({\text{t}}_2^{*}\right)$ or t* is large. This leads to high variance effect estimation as we show in fig. 4 for both causal models. For various variances of t, σ², we plot the mean and standard deviation of RMSE of estimated conditional effect over 10 seeds, against different γ.

Figure 4:

RMSE of estimated conditional effect estimate vs. the strength of confounding γ, for different levels of variance of t, σ². Small σ leads to large conditional estimation error.

Third, we investigate the bias induced due to imperfect estimation of the surrogate intervention in LODE for both causal models. We construct surrogate interventions ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ by ensuring there is confounder-value mismatch $\text{h}(\tilde{\text{t}})\ne \text{h}\left({\text{t}}_2^{*}\right)$. We do this by interrupting Euler integration when the objective ${\mathbb{E}}_{{\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)}{\left(\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2={\delta }^2>0$, where the ${\mathbb{E}}_{{\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)}$ is over our evaluation set upon which we estimate conditional effects. For different α, we plot in fig. 6 the mean and standard deviation of RMSE of estimated conditional effect over 10 seeds, against different degrees of confounder mismatch, δ. The error due to confounder mismatch is mitigated by small α, the Lipschitz-constant of the outcome as a function of h(t). Finally, we consider how step size in Euler integration affects the quality of estimated effects. Large step sizes may result in biased surrogate estimates; this bias is captured in the accumulation error in section 3.1. We focus on the non-linear case in causal model B where gradient errors can accumulate(see appendix A.3.1). We demonstrate this error in fig. 5 where we plot mean and standard deviation of conditional effect RMSE against the strength of confounding, for different step sizes ℓ. We do not report results for larger step sizes (ℓ > 2) because Euler integration diverged for many surrogate estimates.

Figure 6:

RMSE of estimated conditional effect vs. degree of confounder mismatch δ. Error due to conditioning on a mismatched value of the confounder increases with strength of confounding but is mitigated by smoothness of the outcome function.

Figure 5:

RMSE of estimated conditional effect vs. step size in Euler Integrator in causal model B. Accumulating error due to large step size in Euler integrator increases with strength of confounding.

4.2. Effects in Genetics (GWAS)

In this experiment, we explore the associations of genetic factors and Celiac disease. We utilize data from the Wellcome Trust Celiac disease GWAS dataset [8, 6] consisting of individuals with celiac disease, called cases (n = 3796), and controls (n = 8154). We construct our dataset by filtering from the ~ 550,000 SNPs. The only preprocessing in our experiments is linkage disequilibrium pruning of adjacent SNPs (at 0.5 R²) and PLINK [5] quality control. After this, 337,642 SNPs remain for 11,950 people. We imputed missing SNPs for each person by sampling from the marginal distribution of that SNP. No further SNP or person was dropped due to missingness. The objective of this experiment is to show that LODE corrects for confounding and recovers SNPs reported in the literature [8, 25, 14, 1]. To this end, after preprocessing, we included in our data 50 SNPs reported in [8, 25, 14, 1] and 1000 randomly sampled from the rest.

We use outcome models and functional confounders h() traditionally employed in the GWAS literature. We choose a linear $\text{h}(\tilde{\text{t}})={\text{A}}^{\top }\tilde{\text{t}}$, where A is a matrix of the right singular vectors of a normalized Genotype matrix, that correspond to the top 10 singular values [19]. The outcome model is selected from logistic Lasso linear models with various regularization strengths, via cross validation within the training data (60% of the dataset). We defer details about the experimental setup to appendix B.

We then use this outcome model in LODE to compute causal effects on the whole filtered dataset. The effects are computed one SNP at a time. First, for each person $\tilde{\text{t}}$, create ${\tilde{\text{t}}}_{\text{i}}^1,{\tilde{\text{t}}}_{\text{i}}^0$ which correspond to the ith SNP set to 1 and 0 respectively, with all other SNPs same as $\tilde{\text{t}}$. Randomly sample a $\text{h}\left({\text{t}}_2^{*}\right)$ from the marginal p(h(t)) and, using the outcome model P_θ, compute $\varphi (\tilde{\text{t}},\text{i})=\text{log\hspace{0.17em}}{\text{P}}_{\theta }\left(\text{y}=1|{\text{t}}^{\prime }\left({\tilde{\text{t}}}_{\text{i}}^1,\text{h}\left({\text{t}}_2^{*}\right)\right)\right)/{\text{P}}_{\theta }\left(\text{y}=1|{\text{t}}^{\prime }\left({\text{t}}_i^0,\text{h}\left({\text{t}}_2^{*}\right)\right)\right)$. The average effect of SNP i is obtained by averaging across all persons: ${\sum }_{\tilde{\text{t}}}\varphi (\tilde{\text{t}},\text{i})/\text{N}$. Any SNP that beats a specified threshold of effect is deemed relevant to Celiac disease by LODE. We use a 60 − 40% train-test split, and outcome model selection is done via cross-validation within the training set. We did 5-fold cross-validation using just the training set. We use Scikit-learn [18] to fit the outcome models and for cross-validation.

Results

The best outcome model was a Lasso model, trained with regularization constant 10. We select relevant SNPs by thresholding estimated effects at a magnitude > 0.1. From 1050 SNPs (1000 not reported before) LODE returned 31 SNPs, out of which 13 were previously reported as being associated with Celiac disease [8, 25, 14, 1]. In appendix B.2 we plot the true positive and false negative rates of identifying previously reported SNPs, as a function of the effect threshold.

In table 1, we list a few SNPs that were both deemed relevant by LODE and were reported in existing literature [8, 25, 14, 1], their effects, and their Lasso coefficients. The full list is in table 2 in appendix B. If LODE cannot adjust for confounding, the Lasso coefficients would dictate the effects; 0 coefficient means 0 effect. However, the two pairs of SNPs in table 1 show that the effects estimated by LODE do not rely solely on the Lasso coefficients. For the first pair (rs13151961, rs2237236), the effect is the same but the coefficient of one is 0, while the other is positive. We note that rs2237236 was found to be associated with ulcerative colitis [12, 2], which is an inflammatory bowel disease that has been reported to share some common genetic basis with celiac disease [16]. For the second pair, (rs1738074, rs11221332), the magnitude of the effect is smaller for the former, but the coefficient is larger. Thus, LODE adjusts for confounding factors that the outcome model ignored.

Table 1:

A few SNPs previously reported as relevant and recovered by LODE, with estimated effects and Lasso coefficients. LODE produces effect estimates that do not rely purely on the coefficients.

SNP	Effect.	Coef.
rs13151961	0.17	0.32
rs2237236	0.17	0.00
rs1738074	−0.16	−0.23
rs11221332	−0.15	−0.24

5. Discussion

When positivity is violated in traditional OBS-CI, not all effects are estimable without further assumptions. In such cases, practitioners have to turn to parametric models to estimate causal effects. However, parametric models can be misspecified when used without underlying causal mechanistic knowledge. We develop a new general setting of observational causal effect estimation called estimation with functional confounders (EFC) where the confounder can be expressed as a function of the data, meaning positivity is violated. Even when positivity is violated, the effects of many functional interventions are estimable. We develop a sufficient condition called functional positivity (F-POSITIVITY) to estimate effects of functional interventions. Such effects could be of independent interest; like the effect of cumulative dosage of a drug instead of joint effects of multiple dosages at different times.

Second, we prove a necessary condition for nonparametric estimation of effects of the full intervention. We propose the C-REDUNDANCY condition, under which, the effect of the full intervention on t is estimable without parametric restrictions. We develop Level-set Orthogonal Descent Estimation (LODE) that computes surrogate interventions whose effects are estimable and match a conditional effect of interest. Further, we give bounds on errors (theorem 2) induced due to imperfect estimation of the surrogate intervention. Finally, we empirically demonstrate LODE’s ability to correct for confounding in both simulated and real data.

Future.

A few directions of improvement remain which we elaborate next. First, F-POSITIVITY may not hold for all functions g(t) that we want to intervene on. Instead, one could compute a “projection” g_Π to the space of functions that satisfy F-POSITIVITY and inspect the effects defined by g_Π instead. A second direction of interest is to let h(t) only account for a part of the confounding, meaning ignorability is violated. This bias could be mitigated under smoothness conditions of the outcome function and its interaction with the degree of violation of ignorability.

Finally, LODE’s search strategy is Euler integration, which is equivalent to gradient descent with a fixed step size. Optimization techniques like momentum, rescaling the gradient using an adaptive matrix, and using second order hessian information, speed up gradient descent. However, if there are many local or global minima for ${\left(\text{h}(\tilde{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, such techniques will result in a different solution than Euler integration, which could mean that effect estimates are biased. One extension of LODE would allow for search strategies that use such techniques.

Broader Impact.

Our work mainly applies to causal inference where confounders are specified as functions of observed data, such as in problems in genetics and healthcare. We choose to assess the impact of our work through its applications in these fields. A positive impact of the work is that better estimates of causal effects helps guide treatment for people and aid in understanding biological pathways of diseases. However, in healthcare, data collected in hospitals has biases. If, for instance, a certain demographic of people have more complete data collected about them, then this demographic would have better quality effect estimates, potentially meaning that they receive better treatment. This problem could be characterized by evaluating the positivity of treatment and completeness of confounders in electronic health record data split by demographics.

A. Theoretical details

A.1. A note about the assumptions

Note about the assumptions

In theorem 1, assumption 1 consists of three parts that can all be validated on observed data: 1) that the gradient flow converges, 2) that the confounder value of the surrogate matches the confounder value whose effect is of interest, and 3) that the surrogate intervention lies in the support of the pre-outcome variables. Assumption 2 is required for expectations and their gradients to exist and be finite. In theorem 2, assumption 1 requires a consistent estimator of $\mathbb{E}[y|t]$, which can be provided with regression. Assumption 3 lists regularity conditions which help control how the surrogate estimation error propagates to the effect error.

A.2. Proof of Theorem 1

We restate the theorem for completeness:

Theorem 1. Assume C-REDUNDANCY holds. Assuming the following:

1. Let ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ be the limiting solution to the gradient flow equation $\frac{\text{d}\tilde{\text{t}}(\text{s})}{\text{ds}}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, initialized at $\tilde{\text{t}}(0)={\text{t}}^{*}$; i.e. ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)={\text{lim}}_{\text{s}\to \infty }\tilde{\text{t}}(\text{s})$.

   Further, let $\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)=\text{h}\left({\text{t}}_2^{*}\right)$ and ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\in \mathit{\text{supp}}(t)$.

2. $\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$ and $\text{h}(\tilde{\text{t}})$ as functions of $\tilde{\text{t}}$, $\text{h}(\tilde{\text{t}})$ are continuous and differentiable and the derivatives exist for all $\tilde{\text{t}}$, η. Let ${\nabla }_{\tilde{\text{t}}}\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$ exist and be bounded and integrable w.r.t. the probability measure corresponding to p(η), for all values of $\tilde{\text{t}}$ and $\text{h}(\tilde{\text{t}})$.

Then the conditional effect (and therefore the average effect) is identified:

$$\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)\right)=\mathbb{E}\left[y|t={\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right]$$ (10)

Proof. Recall definition of conditional effect $\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)={\mathbb{E}}_{\eta }\text{f}\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)$. Recall ${\nabla }_{\tilde{\text{t}}}$ is the gradient with respect to the first argument of f, that is $\tilde{\text{t}}$. First, by assumption 2, $\mathbb{E}$ and ∇ commute, under the dominated convergence theorem. Then, by C-REDUNDANCY

$${\nabla }_{\tilde{\text{t}}}\varphi {\left(\tilde{\text{t}},\text{h}\left({\text{t}}^{*}\right)\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})={\nabla }_{\tilde{\text{t}}}{\mathbb{E}}_{\eta }\text{f}{\left(\tilde{\text{t}},\text{h}\left({\text{t}}^{*}\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})={\mathbb{E}}_{\eta }\left[{\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\text{t}}^{*}\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})\right]=0.$$

Now consider the gradient flow equation $\text{d}\tilde{\text{t}}(\text{s})/\text{ds}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$. We refer to the gradient evaluated at $\tilde{\text{t}}$ as $\Delta \tilde{\text{t}}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2=-2\left(\text{h}(\tilde{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})$. We will express $\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right)$ as defined by the starting point $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ and the gradient flow equation.

Let the solution path to the gradient flow equation be C with t*, ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ being the starting and ending points respectively. By the Gradient Theorem [26], we have that $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ and $\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right)$ are related via the line integral over C:

$${\int }_{\text{C}}{\nabla }_{\tilde{\text{t}}}\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)\cdot \text{d}\tilde{\text{t}}=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right)-\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)$$

Let $\tilde{\text{t}}(\text{s})$ be a parametrization of solution path C by the scalar time s ∈ [0, ∞). Now, to obtain the value of $\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)$, we will compute the line integral over the vector field defined by ${\nabla }_{\tilde{\text{t}}}\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)$, which exists by assumption 2 in theorem 1, evaluated along the path C defined by $\Delta \tilde{\text{t}}(\text{s})$:

$$\begin{gathered} \varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)+{\int }_{\text{C}}{\nabla }_{\tilde{\text{t}}}\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)\cdot \text{d}\tilde{\text{t}} \\ =\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)+{\int }_0^{\infty }{\nabla }_{\tilde{\text{t}}}\varphi {\left(\tilde{\text{t}}(\text{s}),\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }\frac{\text{d}\tilde{\text{t}}(\text{s})}{\text{\hspace{0.17em}ds}}\text{\hspace{0.17em}ds} \\ =\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)+{\int }_0^{\infty }{\nabla }_{\tilde{\text{t}}}\varphi {\left(\tilde{\text{t}}(\text{s}),\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }\Delta \tilde{\text{t}}(\text{s})\text{\hspace{0.17em}ds} \\ \varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)+{\int }_0^{\infty }-2\left(\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)\right){\nabla }_{\tilde{\text{t}}}\varphi {\left(\tilde{\text{t}}(\text{s}),\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}}(\text{s}))\text{\hspace{0.17em}ds} \\ =\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)+0\{\text{by\hspace{0.17em}C-REDUNDANCY\hspace{0.17em}}\} \end{gathered}$$ (11)

Finally, by assumption 1 in theorem 1, $\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)=\text{h}\left({\text{t}}_2^{*}\right)$, and so

$$\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)\right)$$ (12)

For clarity, the same equation, but using t′ and suppressing dependence on t*, $\text{h}\left({\text{t}}_2^{*}\right))$:

$$\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}^{\prime }\right)\right)$$ (13)

Under the causal model for EFC, the outcome y = f(t, h(t), η). Then, $\forall \tilde{\text{t}}\in \text{supp}(\text{p}(t))$,

$$\mathbb{E}[y|t=\tilde{\text{t}}]={\mathbb{E}}_{\eta }[\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )]=\varphi (\tilde{\text{t}},\text{h}(\tilde{\text{t}})).$$ (14)

Using that ${\text{t}}^{\prime }\left({\text{t}}^{*},{\text{t}}_2^{*}\right)\in \text{supp}(\text{p}(t))$ and eqs. (13) and (14), the conditional effect is identified

$$\begin{gathered} \varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right),\text{h}\left({\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)\right) \\ =\mathbb{E}\left[y|t={\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right] \end{gathered}$$ (15)

Thus, the conditional effect, and consequently the average effect, are identified as $\mathbb{E}\left[y|{\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right]$ and $\tau \left({\text{t}}^{*}\right)={\mathbb{E}}_{\text{h}(t)}\mathbb{E}\left[y|{\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}(t)\right)\right]$ respectively.

Note about convergence of gradient flow

Any ODE’s solution, if it exists and converges, converges to an ω-limit set [27]. An ω-limit set is nonempty when the solution path lies entirely in a closed and bounded set and can consist of limit cycles, equilibrium points, or neither [13, 27]. A gradient flow equation $\text{d}\tilde{\text{t}}(\text{s})/\text{ds}=-\nabla \text{h}(\tilde{\text{t}})$ (also called a gradient system) has the special property that its ω-limit set only consists of critical points of $\text{h}(\tilde{\text{t}})$; critical points of $\text{h}(\tilde{\text{t}})$ are also equilibrium points of the gradient flow equation [13]. Further, if $\nabla \text{h}(\tilde{\text{t}})$ exists and is bounded and $\text{h}(\tilde{\text{t}})$ has bounded sublevel sets $(\{\tilde{\text{t}}:\text{h}(\tilde{\text{t}})⩽\text{c}\})$, then the solution to the gradient flow equation will entirely lie within a bounded set. This is because along the solution path, $\text{h}(\tilde{\text{t}}(\text{s}))$ always decreases meaning that the solution will remain in any sublevel set it started in. Thus, if $\text{h}(\tilde{\text{t}})$ has bounded sublevel sets, the solution of the gradient flow equation will converge only to critical points of $\text{h}(\tilde{\text{t}})$.

A.3. Estimation error in LODE

Theorem 2. Consider the conditional effect $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$. Let $\widehat{\text{t}}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ be the estimate of the surrogate intervention computed by LODE, computed via Euler integration of the gradient flow $\frac{\text{d}\tilde{\text{t}}(\text{s})}{\text{ds}}=-{\nabla }_{\tilde{\text{t}}}{\left(\text{h}(\tilde{\text{t}}(\text{s}))-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, initialized at $\tilde{\text{t}}(0)={\text{t}}^{*}$. Assume the true surrogate ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$ exists and is the limiting solution to the gradient flow equation.

1. Let the finite sample estimator of $\mathbb{E}[y|t=\tilde{\text{t}}]$ be $\widehat{\text{f}}(\tilde{\text{t}})$. Let the error for all $\tilde{\text{t}}$ be bounded, $|\widehat{\text{f}}(\tilde{\text{t}})-\mathbb{E}[y|t=\tilde{\text{t}}]|⩽\text{c}(\text{N})$, where N is the sample size and lim_N→∞ c(N) = 0.

2. Assume K Euler integrator steps were taken to find the surrogate estimate $\widehat{\text{t}}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, each of size ℓ. Let the maximum confounder mismatch be ${\text{max}}_{\text{i}⩽\text{K}}{\left(\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2=\text{M}$.

3. Let ${\text{L}}_{z,\tilde{\text{t}}}$ be the Lipschitz-constant of $\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)$ as a function of $\text{h}\left({\tilde{\text{t}}}_2\right)$, for fixed $\tilde{\text{t}}$.

   Let L_e be the Lipschitz-constant of $\mathbb{E}[y|t=\tilde{\text{t}}]=\varphi (\tilde{t},\text{h}(\tilde{\text{t}}))$ as a function of $\tilde{\text{t}}$.

   Assume h has a gradient with bounded norm, $\parallel \nabla \text{h}(\tilde{\text{t}}){\parallel }_2<{\text{L}}_{\text{h}}$.

   Assume f’s Hessian has bounded eigenvalues: $\forall \tilde{\text{t}}$, ${\tilde{\text{t}}}_2$, ${\nabla }_{\tilde{\text{t}}}^2\varphi {\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)}_2⩽{\sigma }_{\text{H}\varphi }$.

The conditional effect estimate error, $\xi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=|\widehat{\text{f}}(\widehat{\text{t}})-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|$, is upper bounded by:

$$\text{c}(\text{N})+\text{min\hspace{0.17em}}\left({\text{L}}_{\text{e}}{\Vert {\text{t}}^{\prime }-\widehat{\text{t}}\Vert }_2,2\text{K}{\ell }^2\left(\mathcal{O}(\ell )+\text{M}{\sigma }_{\text{H}\varphi }{\text{L}}_{\text{h}}^2\right)+{\text{L}}_{z,\widehat{\text{t}}}{\Vert \text{h}(\widehat{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\Vert }_2\right)$$ (16)

Proof. (of Theorem 2) Recall the definition of conditional effect : $\varphi \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)={\mathbb{E}}_{\eta }\text{f}\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)$.

LODE’s estimate of the conditional effect is $\widehat{\text{f}}\left(\widehat{\text{t}}\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\right)$. We will suppress notation for dependence on t*, $\text{h}\left({\text{t}}_2^{*}\right)$ and use t′ and $\widehat{\text{t}}$ to refer to the true surrogate intervention and the estimated surrogate interventions respectively. Note $\widehat{\text{f}}$ is the estimate of the conditional expectation $\mathbb{E}[y|t=\tilde{\text{t}}]$, learned from N samples. We first bound the error by splitting into two parts and bounding each separately:

$$\begin{gathered} |\xi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|=|\widehat{\text{f}}(\widehat{\text{t}})-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)| \\ ⩽|\widehat{\text{f}}(\widehat{\text{t}})-\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))|+|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)| \\ ⩽\text{c}(\text{N})+|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)| \\ ⩽|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left(\widehat{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)|+|\varphi \left(\widehat{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|+\text{c}(\text{N}) \end{gathered}$$

The first term is bounded via the Lipschitz-ness of ϕ as a function of $\text{h}(\tilde{\text{t}})$ with fixed first argument $\tilde{\text{t}}=\widehat{\text{t}}$.

$$|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left(\widehat{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)|⩽L_{z,\widehat{\text{t}}}|\text{h}(\widehat{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)|$$

We now bound the remaining term. Recall that LODE’s computation of the surrogate intervention involved K gradient steps, each of size ℓ. We work with a constant step-size but the analysis can be generalized to a non-uniform step size. Indexing steps with i, let ${\text{d}}_{\text{i}}=\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)$ be the confounder mismatch error at the ith iterate. Then note that $\widehat{\text{t}}={\text{t}}^{*}-\ell {\sum }_{\text{i}=0}^{\text{K}-1}2{\text{d}}_{\text{i}}{\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)$. We can use this to bound the error $\varphi \left(\widehat{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$. With ${\tilde{\text{t}}}_{\text{K}}=\widehat{\text{t}}$ and ${\tilde{\text{t}}}_0={\text{t}}^{*}$, we proceed by expressing the error as a telescoping sum and using the Taylor expansion for $\varphi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)$ in terms of the the first argument $\tilde{\text{t}}$.

$$\varphi \left(\widehat{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\sum _{\text{i}=0}^{\text{K}-1}\varphi \left({\tilde{\text{t}}}_{\text{i}+1},\text{h}\left({\text{t}}_2^{*}\right)\right)-\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right)$$ (17)

$$=\sum _{\text{i}=0}^{\text{K}-1}{\nabla }_{\tilde{\text{t}}}\varphi {\left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }\left({\tilde{\text{t}}}_{\text{i}+1}-{\tilde{\text{t}}}_{\text{i}}\right)$$ (18)

$$+\frac{1}{2}{\left({\tilde{\text{t}}}_{\text{i}+1}-{\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right)\left({\tilde{\text{t}}}_{\text{i}+1}-{\tilde{\text{t}}}_{\text{i}}\right)+\mathcal{O}\left({\Vert {\tilde{\text{t}}}_{\text{i}+1}-{\tilde{\text{t}}}_{\text{i}}\Vert }_2^3\right)$$ (19)

$$=\sum _{\text{i}=0}^{\text{K}-1}2\ell {\text{d}}_{\text{i}}{\nabla }_{\tilde{\text{t}}}\varphi {\left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)+2{\left(\ell {\text{d}}_{\text{i}}\right)}^2{\nabla }_{\tilde{\text{t}}}\text{h}{\left({\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)+\mathcal{O}\left({\ell }^3\right)$$ (20)

$$=\sum _{\text{i}=0}^{\text{K}-1}0+2{\left(\ell {\text{d}}_{\text{i}}\right)}^2{\nabla }_{\tilde{\text{t}}}\text{h}{\left({\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)+\mathcal{O}\left({\ell }^3\right)$$ (21)

$$=\mathcal{O}\left(\text{K}{\ell }^3\right)+\sum _{\text{i}=0}^{\text{K}-1}2{\left(\ell {\text{d}}_{\text{i}}\right)}^2{\nabla }_{\tilde{\text{t}}}\text{h}{\left({\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)$$ (22)

$$⩽\mathcal{O}\left(\text{K}{\ell }^3\right)+\sum _{\text{i}=0}^{\text{K}-1}2{\left(\ell \left(\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)\right)}^2|{\nabla }_{\tilde{\text{t}}}\text{h}{\left({\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)|$$ (23)

$$⩽\mathcal{O}\left(\text{K}{\ell }^3\right)+\sum _{\text{i}=0}^{\text{K}-1}2{\ell }^2\text{M}|{\nabla }_{\tilde{\text{t}}}\text{h}{\left({\tilde{\text{t}}}_{\text{i}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}^2\varphi \left({\tilde{\text{t}}}_{\text{i}},\text{h}\left({\text{t}}_2^{*}\right)\right){\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)|$$ (24)

$$⩽\mathcal{O}\left(\text{K}{\ell }^3\right)+\sum _{\text{i}=0}^{\text{K}-1}2{\ell }^2\text{M}{\sigma }_{\text{H}\varphi }{\Vert {\nabla }_{\tilde{\text{t}}}\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)\Vert }_2^2$$ (25)

$$⩽\mathcal{O}\left(\text{K}{\ell }^3\right)+\sum _{\text{i}=0}^{\text{K}-1}2{\ell }^2\text{M}{\sigma }_{\text{H}\varphi }{\text{L}}_{\text{h}}^2$$ (26)

$$=2\text{K}{\ell }^2\left(\mathcal{O}(\ell )+\text{M}{\sigma }_{\text{H}\varphi }{\text{L}}_{\text{h}}^2\right),$$ (27)

where the inequalities follow by the maximum value of ${\left(\text{h}\left({\tilde{\text{t}}}_{\text{i}}\right)-\text{h}\left({\text{t}}_2^{*}\right)\right)}^2$, bounded eigenvalues of the Hessian of ϕ and the Lipschitz-ness of $\text{h}(\tilde{\text{t}})$.

Another way we bound the error is via the Lipschitz constant of the conditional expectation as a function of $\tilde{\text{t}}$. Recall this is L_e. An alternate bound on the error is as follows:

$$|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|=|\varphi (\widehat{\text{t}},\text{h}(\widehat{\text{t}}))-\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}^{\prime }\right)\right)|⩽{\text{L}}_{\text{e}}{\Vert {\text{t}}^{\prime }-\widehat{\text{t}}\Vert }_2$$

The bound follows:

$$|\xi \left(\tilde{\text{t}},\text{h}\left({\text{t}}_2^{*}\right)\right)|⩽\text{c}(\text{N})+\text{min\hspace{0.17em}}\left({\text{L}}_{\text{e}}{\Vert {\text{t}}^{\prime }-\widehat{\text{t}}\Vert }_2,2\text{K}{\ell }^2\left(\mathcal{O}(\ell )+\text{M}{\sigma }_{\text{H}\varphi }{\text{L}}_{\text{h}}^2\right)+{\text{L}}_{z,\widehat{\text{t}}}{\Vert \text{h}(\widehat{\text{t}})-\text{h}\left({\text{t}}_2^{*}\right)\Vert }_2\right)$$

A.3.1. A note on linear confounder functions and LODE

In the proof above, the error in Euler integration accumulates due to terms like this one: ${\nabla }_{\tilde{\text{t}}}^{\top }\text{h}(\tilde{\text{t}}){\nabla }_{\tilde{\text{t}}}^2\text{f}\left(\tilde{\text{t}},\text{h}\left({\text{t}}^{*}\right),\eta \right){\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})$. For a linear confounder function that satisfies ${\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=\beta$, such terms can be expressed as ${\beta }^{\top }{\nabla }_{\tilde{\text{t}}}\left({\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\text{t}}^{*}\right),\eta \right)}^{\top }\beta \right)={\beta }^{\top }{\nabla }_{\tilde{\text{t}}}(0)=0$ under C-REDUNDANCY. Thus, such error does not accumulate even with large step sizes.

Further, note that the gradient flow equation in LODE for the causal model A in section 4 is a linear ODE whose solution has a closed form expression and one can estimate the surrogate without numerical integration [27].

A.4. Proof of sufficiency of Effect Connectivity

Theorem 3. Under Effect Connectivity, eq. (9), any surrogate intervention ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)\in \mathit{\text{supp}}(t)$.

Proof. Recall $\varphi (\tilde{\text{t}},\text{h}(\tilde{\text{t}}))={\mathbb{E}}_{\eta }\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}),\eta )$. We have ∀t* ∈ supp(p(t)):

$$\text{p}\left(\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0\Rightarrow \text{p}\left(\varphi (t,\text{h}(t))=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)|\text{h}(t)=\text{h}\left({\text{t}}_2^{*}\right)\right)>0.$$

This implies Ǝt′ ∈ supp(t), $\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, s.t. $\text{h}\left({\text{t}}^{\prime }\right)=\text{h}\left({\text{t}}_2^{*}\right)$.

Then, $\varphi \left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}_2^{*}\right)\right)=\varphi \left({\text{t}}^{\prime },\text{h}\left({\text{t}}^{\prime }\right)\right)=\mathbb{E}\left[y|t={\text{t}}^{\prime }\right]$.

A.5. Necessity of Effect Connectivity for Nonparametric effect estimation in EFC

Theorem 4. Effect Connectivity is necessary for nonparametric effect estimation in EFC.

Proof. (Proof of Theorem 4) Let the outcome be y = f(t, h(t)). Recall the joint distribution p(t, y) and let h(t) be the confounder. Let Effect Connectivity be violated, i.e. there exists a non-measure-zero subset B ∈ supp(t) × supp(h(t)) such that⁶:

$$\forall \tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\in \text{B},\text{\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}\hspace{0.17em}p}\left(\text{f}(t,\text{h}(t))=\text{f}\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)|\text{h}(t)=\text{h}\left({\tilde{\text{t}}}_2\right)\right)=0.$$

Now, we construct a new outcome y₂ = f₂(t, h(t)) and show the conditional effects for this new outcome are different from the one defined by f on $\forall \left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)\in \text{B}$. Let

$${\text{f}}_2\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)=\text{f}\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)+10*1\left(\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)\in \text{B}\right)|.$$

We have ${\text{f}}_2(\tilde{\text{t}},\text{h}(\tilde{\text{t}}))=\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}))\forall \tilde{\text{t}}\in \text{supp}(t)$ as the additional term in f₂ is only present for $\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)\in \text{B}$; this follows from the fact that $\forall \tilde{\text{t}}\in \text{supp}(t)$, $\left(\tilde{\text{t}},\text{h}\left(\tilde{\text{t}}\right)\right)\notin \text{B}$ as

$$\text{p}[\text{f}(t,\text{h}(t))=\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}))|\text{h}(t)=\text{h}(\tilde{\text{t}})]=\text{p}[\text{f}(t,\text{h}(t))=\text{f}(\tilde{\text{t}},\text{h}(\tilde{\text{t}}))]>0.$$

Thus, p(y, t) =^d p(y₂, t) are equal in distribution since B ∩ supp(t, h(t)) = ∅. This means that the conditional effects are different for the outcomes y, y₂ for all $\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right)\right)\in \text{B}$:

$$\mathbb{E}\left[y|\text{do}(t=\tilde{\text{t}}),\text{h}(t)=\text{h}\left({\tilde{\text{t}}}_2\right)\right]\ne \mathbb{E}\left[y_2|\text{do}(t=\tilde{\text{t}}),\text{h}(t)=\text{h}\left({\tilde{\text{t}}}_2\right)\right]$$

Therefore, for causal models that violates Effect Connectivity, there exist observationally equivalent causal models with different causal effects. Thus, nonparametric effect estimation is impossible. Thus, Effect Connectivity is required for EFC.

A.6. Algorithmic details

We give in algorithm 1 pseudocode for LODE.

Extensions of LODE

Consider that we have access to m(h(t)) for some bijective differentiable function m(·), instead of h(t). The orthogonality in C-REDUNDANCY holds ${\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{m}(\text{h}(\tilde{\text{t}}))={\text{m}}^{\prime }(\text{h}(\tilde{\text{t}})){\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0$. Then, using $\text{m}(\text{h}(\tilde{\text{t}}))$ to compute the surrogate ${\text{t}}^{\prime }\left({\text{t}}^{*},\text{h}\left({\text{t}}_2^{*}\right)\right)$, LODE would estimate valid effects. Similarly, LODE can estimate the effect on any differentiable transformation of the outcome m(y), because ${\nabla }_{\tilde{\text{t}}}\text{m}{\left({\text{y}}_{\tilde{\text{t}}}\right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})={\text{m}}^{\prime }\left({\text{y}}_{\tilde{\text{t}}}\right){\nabla }_{\tilde{\text{t}}}\text{f}{\left(\tilde{\text{t}},\text{h}\left({\tilde{\text{t}}}_2\right),\eta \right)}^{\top }{\nabla }_{\tilde{\text{t}}}\text{h}(\tilde{\text{t}})=0$ holds.

B. Experimental Details

B.1. Functional confounders in GWAS

Here, we show how h(t) = At and A reflect the traditional PCA based adjustment in GWAS. Recall population structure acts as a confounder in GWAS. Price et al. [19] demonstrated that using the principal components of the normalized genetic relationships matrix adjusts for confounding due to population structure in GWAS. Let the genotype matrix be G with people as rows and SNPs as columns, such that each element is one of 0, ¹/2, 1, where ¹/2 and 1 refer to one and two copies of the allele respectively at the position of the SNP. With p_s as the allele frequency at SNP s [28], Φ is the genetic relationship matrix whose elements are defined as ${\Phi }_{\text{i},\text{j}}=\frac{1}{\text{S}}{\sum }_{\text{s}=1}^{\text{S}}\left({\text{G}}_{\text{i},\text{s}}-{\text{p}}_{\text{s}}\right)\left({\text{G}}_{\text{j},\text{s}}-{\text{p}}_{\text{s}}\right)/{\text{p}}_{\text{s}}\left(1-{\text{p}}_{\text{s}}\right)$. Then, Price et al. [19] compute the top K (10 suggested) principal components of Φ to use as the axes of variation due to the population structure. The eigenvectors of Φ are the left eigenvectors of $\widehat{\text{G}}$ such that $\Phi ={\widehat{\text{G}}\widehat{\text{G}}}^{\top }$ which capture independent axes of variation of individuals.

Price et al. [19] exploit the idea that if a SNP aligns with some of the axes of variation, this is due to the population structure. These axes of variation are the top K eigenvectors U of $\varphi ={\widehat{\text{G}}\widehat{\text{G}}}^{\top }\approx \text{U}\wedge {\text{U}}^{\top }$, where $\text{U}\in {ℝ}^{\text{N}\times \text{K}}$, $\Phi \in {ℝ}^{\text{N}\times \text{N}}$ and $\Lambda \in {ℝ}^{\text{K}\times \text{K}}$. Here, U are also the left singular vectors of $\widehat{\text{G}}\approx {\text{UΣV}}^{\top }$ where $\Sigma \in {ℝ}^{\text{K}\times \text{K}}$ is diagonal, and $\text{V}\in {ℝ}^{\text{S}\times \text{K}}$. We use ≈ to denote that the chosen K eigenvectors explain the variation due to population structure; what remains are random mutations.

Let the sth SNP be ${\widehat{\text{G}}}_{\cdot ,\text{s}}\in {ℝ}^{\text{N}}$, which is a column in $\widehat{\text{G}}$. In Price et al. [19], population structure in the sth SNP is captured in ${\widehat{\text{G}}}_{\cdot ,\text{s}}^{\top }\text{U}$. In words, projecting the SNP ${\widehat{\text{G}}}_{\cdot ,\text{s}}$ onto the axes of variation in individuals gives the population structure between sth SNP and the outcome. This projection ${\widehat{\text{G}}}_{\cdot ,\text{s}}^{\top }\text{U}$ is a row of ${\widehat{\text{G}}}^{\top }\text{U}\in {ℝ}^{\text{S}\times \text{K}}$. In turn, ${\widehat{\text{G}}}^{\top }\text{U}\in {ℝ}^{\text{S}\times \text{K}}$ is the population structure in all SNPs. Projecting this population structure onto the genotype of an individual gives the confounding due to population structure amongst the SNPs present in the genotype. With ${\text{G}}_{\text{j},\cdot }\in {\left\{0,\mathrm{½},1\right\}}^{\text{S}}$ as the genotype for an individual j, this projection is $\left({\left({\widehat{\text{G}}}^{\top }\text{U}\right)}^{\top }{\text{G}}_{\text{j,}\cdot }\right)$. However, $\widehat{\text{G}}\approx {\text{UΣV}}^{\top }$ implies that ${\widehat{\text{G}}}^{\top }\text{U}\approx \text{VΣ}$. Reflecting this, h(t) = ΣV^T t is the functional confounder for an individual t.

B.2. Expanded results

In table 2, we list the 13 SNPs recovered by LODE, that have been previously reported as relevant to Celiac disease. In fig. 7, we plot the true positive and false negative rate amongst SNPs deemed relevant by LODE. The ground truth here are the SNPs reported associated with celiac disease in prior literature.

Figure 7:

True positive vs. False negative rate as we vary the threshold on average effects, that determines which SNPs LODE deems relevant to the outcome.

Table 2:

Full list of SNPs previously reported as relevant that were recovered by LODE, and their estimated effects and Lasso coefficients for SNPs. The effect threshold here is 0.1.

SNP	Effect	Lasso Coef.
rs3748816	0.12	0.20
rs10903122	0.10	0.17
rs2816316	0.11	0.20
rs13151961	0.17	0.32
rs2237236	0.17	0.00
rs12928822	0.14	0.29
rs2187668	−0.70	−2.37
rs2327832	−0.12	−0.20
rs1738074	−0.16	−0.23
rs11221332	−0.15	−0.24
rs653178	−0.13	−0.21
rs4899260	−0.12	−0.19
rs17810546	−0.12	−0.20