Unmeasured Confounding for General Outcomes, Treatments, and Confounders

Abstract

Uncontrolled confounding in observational studies gives rise to biased effect estimates. Sensitivity analysis techniques can be useful in assessing the magnitude of these biases. In this paper, we use the potential outcomes framework to derive a general class of sensitivity-analysis formulas for outcomes, treatments, and measured and unmeasured confounding variables that may be categorical or continuous. We give results for additive, risk-ratio and odds-ratio scales. We show that these results encompass a number of more specific sensitivity-analysis methods in the statistics and epidemiology literature. The applicability, usefulness, and limits of the bias-adjustment formulas are discussed. We illustrate the sensitivity-analysis techniques that follow from our results by applying them to 3 different studies. The result bias formulas are particularly simple and easy to use in settings in which the unmeasured confounding variable is binary with constant effect on the outcome across treatment and covariate levels, and with a constant prevalence difference across covariate levels when comparing 2 treatment levels.

Unmeasured confounders in observational studies result in biased effect estimates. Several sensitivity-analysis and bias-modeling techniques have now been developed to handle uncontrolled confounding.^1–22 Although the literature is large and it would be difficult to provide a comprehensive review, a number of the existing techniques are restricted to simple or very particular settings. There is also a literature on bounds for causal effects or “partial identification”^23–27 that makes fewer assumptions than sensitivity-analysis techniques, but that thereby effectively considers extreme scenarios which in some settings give bounds including the null irrespective of the data.

In this paper, we extend the bias-modeling and sensitivity-analysis literature on uncontrolled confounding. Using the potential outcomes framework, we derive a general class of formulas for sensitivity analysis of uncontrolled confounding with outcomes, treatments, and measured and unmeasured confounders that may be categorical (2 or more categories) or continuous. The formulas generalize many previous results in the literature and give rise to very flexible sensitivity-analysis techniques that can be used in a wide range of applications. We also describe a particularly easy-to-use sensitivity-analysis technique that can be used under some simplifying assumptions that follows from our results and is summarized in the discussion section.

Notation, Definitions, and Assumptions

We will use the potential outcomes or counterfactual framework.^28–30 Let treatment A denote the treatment received by a particular individual. Let Y be the observed post-treatment outcome of that individual. Let Y_a denote the potential outcome Y for an individual if the treatment A, perhaps contrary to fact, had been set to value a. Note that we assume that the potential outcome Y_a for an individual does not depend on the treatments received by other individuals. This assumption is sometimes referred to as SUTVA, the stable unit treatment value assumption³⁰ or as a no-interference assumption.³¹ Furthermore, we require the consistency assumption that Y_A = Y, ie, that the value of Y that would have been observed if A had been set to what it in fact was is equal to the value of Y which was actually observed. Therefore, the only potential outcome for an individual that we observe is the potential outcome Y_A, the value of Y that would have been observed if A were set to what it in fact was. We will use the notation S∐T∣V to denote that a variable S is independent of another variable T conditional on a third variable V. To simplify notation further, we will use E(S∣t) to denote E(S∣T = t) and P(s∣t) to denote P(S = s∣T = t).

Let X denote observed categorical or continuous covariates. Let U denote an unmeasured categorical or continuous confounding variable or variables. Suppose that Y is dichotomous, ordinal, or continuous and that A is categorical, ordinal, or continuous. For causal contrasts, we compare expected potential outcomes (ie, counterfactual outcomes) for any 2 treatment levels, a₁ and a₀, of A where a₀ is taken as the reference.

The average causal effect in the total population and among those receiving treatment A = a₁ or A = a₀ are given respectively by E(Y_a1) − E(Y_a0), E(Y_a1∣a₁ − E(Y_a0∣a₁),and E(Y_a1∣a₀) − E(Y_a0∣a₀). Suppose that the effect of A on Y is unconfounded given (U, X), where again U is unmeasured; ie, in counterfactual notation we assume that Y_a∐A∣X,U. We then have that the true causal effects are given by adjusting for both X and U:

$$E(Y_{a_1})-E(Y_{a_0})={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_0,x,u)\}P(u\mid x)P(x)$$

$$E(Y_{a_1}\mid a_1)-E(Y_{a_0}\mid a_1)={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_0,x,u)\}P(u\mid x,a_1)P(x\mid a_1)$$

$$E(Y_{a_1}\mid a_0)-E(Y_{a_0}\mid a_0)={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_0,x,u)\}P(u\mid x,a_0)P(x\mid a_0)$$

If adjustment is made for X but not U, we would obtain the following expressions for the average outcome differences adjusted for X when the target population is the total group, or those exposed to a₁ or a₀, respectively:

$${\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x)$$

$${\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x\mid a_1)$$

$${\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x\mid a_0)$$

The bias due to not controlling for the unmeasured confounder U is thus given by the difference between the observed average outcome differences, adjusted for X, and the true causal effect. Let d_a+, d_a1, and d_a0 denote the relevant bias when the target population is the total group, or those exposed to a₁ or a₀, respectively:

$$d_{a_{+}}={\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x)-\{E(Y_{a_1})-E(Y_{a_0})\}$$

$$d_{a_1}={\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x\mid a_1)-\{E(Y_{a_1}\mid a_1)-E(Y_{a_0}\mid a_1)\}$$

$$d_{a_0}={\sum }_x\{E(Y\mid a_1,x)-E(Y\mid a_0,x)\}P(x\mid a_0)-\{E(Y_{a_1}\mid a_0)-E(Y_{a_0}\mid a_0)\}.$$

In the next section, we derive formulas for these biases that can be used in sensitivity analysis.

General Bias Formulas for Sensitivity Analysis for Average Treatment Effects

Because we do not have data on U, we cannot obtain unbiased estimates of causal effects. We can only estimate the average outcome differences, adjusted for the observed covariates X. We can proceed, however, by using sensitivity analysis. The following theorem is a generalization of previous work.^1,8,21 Complete proofs of all results are given in the Appendix and eAppendix (http://links.lww.com/EDE/A429). The result gives formulas for the bias d_a+, d_a1, and d_a0 in terms of the relationship between the unmeasured confounder(s) U and the outcome Y as well as the relationship between U and treatment group A.

Theorem 1. If Y_a∐A∣X, U, and if u′ is any chosen reference value for the unmeasured confounder U then

$$d_{a_{+}}={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_1,x,u\prime )\}\{P(u\mid a_1,x)-P(u\mid x)\}P(x)-{\sum }_x{\sum }_u\{E(Y\mid a_0,x,u)-E(Y\mid a_0,x,u\prime )\}\{P(u\mid a_0,x)-P(u\mid x)\}P(x)$$

$$d_{a_1}={\sum }_x{\sum }_u\{E(Y\mid a_0,x,u)-E(Y\mid a_0,x,u\prime )\}\{P(u\mid a_1,x)-P(u\mid a_0,x)\}P(x\mid a_1)$$

$$d_{a_0}={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_1,x,u\prime )\}\{P(u\mid a_1,x)-P(u\mid a_0,x)\}P(x\mid a_0).$$

To use these bias formulas in sensitivity analysis for the overall causal effect E(Y_a1) − E(Y_a0), one would need to specify (i) the relation between U and Y, among those with treatment level A = a₁ and A = a₀, within each stratum of X, ie, {E(Y∣a₁, x, u) − E(Y∣a₁, x, u′)} and {E(Y∣a₀, x, u) − E(Y∣a₀, x, u′)}, and (ii) how the distribution of the unmeasured confounder U among those with treatment level A = a₁ and A = a₀ compares with the overall distribution of U, within each stratum of X, ie, {P(u∣a₁, x) − P(u∣x)} and {P(u∣a₀, x) − P(u∣x)}. One could then use the observed data to estimate outcome differences adjusted for X only, Σ_x{E(Y∣a₁, x) − E(Y∣a₀, x)}P(x), one could use the sensitivity-analysis specification concerning (i) and (ii) to estimate the bias d_a+ and then one could produce an estimate of E(Y_a1) − E(Y_a0) by subtracting the bias d_a+ from the estimate adjusted for X only. As can be seen for the bias formulas for d_a1 and d_a0, if one wishes to estimate the causal effect only among those with A = a₁ or A = a₀ then fewer specifications need to be made in the sensitivity analysis.

The result applies to mean differences in binary, ordinal or continuous outcomes for categorical or continuous treatments, with categorical or continuous measured and unmeasured confounding variables. We note that if one or both of X and U are continuous rather than categorical, the sums in Theorem 1 can be replaced with appropriate integrals. Theorem 1 is quite general, and in the Appendix we discuss how it encompasses a number of more specific sensitivity-analysis techniques in the literature.^1,6,8,11,21 Unlike many of the existing techniques, the current approach does not assume that the unmeasured confounders are independent of the measured confounders (see the section in the Appendix on the relation to the external adjustment literature for more detail). The Appendix also discusses other parameterizations for sensitivity analysis (see the section on the relationship to Rosenbaum and Rubin⁶). We discuss standard errors and confidence limits for these bias-adjusted estimates below.

Note that, although the result in Theorem 1 provides a very general approach to sensitivity analysis, it requires that a great deal of information be specified; namely, on (i) the relationship between the unmeasured variable U and the outcome Y across strata of treatment and the measured covariates and also (ii) for each stratum of the measured covariates, how the distribution of the unmeasured confounder U among those with various treatment levels compares with the overall distribution of U. Thus for each stratum of the measured covariates one would have to specify 4 pieces of information (ie, (i) and (ii) for the 2 different treatment levels); if there are many strata of the measured covariates, this will not be particularly feasible in practice. Below we show that under some simplifying assumption, the formulas in Theorem 1 reduce to expressions that are easier to use in practice and that require the specification of fewer sensitivity-analysis parameters.

There is, in general, a tension between generality and complexity. A general approach makes fewer assumptions but requires one to specify a large number of sensitivity parameters; a simpler approach as described below is easier to use and requires the specification of fewer parameters, but it makes stronger simplifying assumptions. Depending on the strength of assumptions a researcher is willing to make concerning the unmeasured confounders, more or less complex approaches are possible in the application of Theorem 1. In the applications section below, we illustrate both a particularly simple sensitivity analysis and another which makes more use of the generality of the bias formulas in Theorem 1. We now turn to a discussion of the sensitivity-analysis approach that results under simplifying assumptions.

If one makes the simplifying assumption that the relationship between U and Y, ie, E(Y∣a, x, u) − E(Y∣a, x, u′), does not vary between strata of A then the expression for d_a+ simplifies to

$$d_{a_{+}}={\sum }_x{\sum }_u\{E(Y\mid a,x,u)-E(Y\mid a,x,u\prime )\}\{P(u\mid a_1,x)-P(u\mid a_0,x)\}P(x).$$

If, in addition, the relationship between U and A, namely {P(u∣a₁, x) − P(u∣a₀, x)}, also does not vary between strata of X, then this expression further simplifies to: Σ_u{E(Y∣a, x, u) − E(Y∣a, x, u′)}{P(u∣a₁, x) − P(u∣a₀, x)}. If U is binary, it further reduces to {E(Y∣a, x, U = 1) − E(Y∣a, x, U = 0)}{P(U = 1∣a₁, x) − P(U = 1∣a₀, x)}. Thus a straightforward sensitivity-analysis technique under simplifying assumptions would consist of hypothesizing a binary confounding variable U with a constant prevalence difference, δ = {P(U = 1∣a₁, x) − P(U = 1∣a₀, x)}, comparing treatment levels a₁ and a₀ over strata of X, and with E(Y∣a, x, U = 1) − E(Y∣a, x, U = 0) = γ constant over strata of A and X; the magnitude of the bias comparing the true causal effect E(Y_a1) − E(Y_a0) with the estimate adjusted only for X but not U, Σ_x {E(Y∣a₁, x) − E(Y∣a₀, x))P(x), is then simply the product δγ, ie,

$$d_{a_{+}}=\delta \gamma .$$

This simple formula has been obtained previously^11,32 but under much stronger assumptions (see Appendix for further discussion). Unlike many other results in the sensitivity-analysis literature, this simple sensitivity analysis (following from Theorem 1) does not presuppose that any particular method, model, or functional form assumption was used to obtain the initial estimate adjusted only for X and not U. Its generality is illustrated in the application section below.

Note that the relationship between U and A, ie, E(Y∣a, x, u) − E(Y∣a, x, u′), may or may not allow for an interpretation as the causal effect of U on Y. If the effect of U on Y is unconfounded given X, then E(Y∣a, x, u) − E(Y∣a, x, u′) may allow for an interpretation as a causal effect of U on Y—namely, as the direct effect of U on Y controlling for A (see the eAppendix [http://links.lww.com/EDE/A429] for the precise conditions for such an interpretation). However, the assumption that the effect of A on Y is unconfounded given (U, X) does not necessarily imply that the effect of U on Y is unconfounded given X. We note, nevertheless, that the expression E(Y∣a, x, u) − E(Y∣a, x, u′) does not need to have a causal interpretation to use the bias formulas in Theorem 1 for sensitivity analysis and external adjustment. Causal diagrams can clarify when an expression such as E(Y∣a, x, u) − E(Y∣a, x, u′) can be interpreted causally.³³ Further discussion of these issues is given in the eAppendix (http://links.lww.com/EDE/A429).

We note also that bias formulas analogous to those in Theorem 1 also hold for conditional causal effects. The conditional causal effect with the entire population, or treatment level a₁ or a₀ taken as the standard, are given by {E(Y_a1∣x) − E(Y_a0∣x)}, {E(Y_a1∣a₁, x) − E(Y_a0∣a₁,x)}, and {E(Y_a1∣a₀, x) − E (Y_a0∣a₀, x)}, respectively. If we let d_a+ (x), d_a1 (x), and d_a0 (x) denote the difference between the true effects and the estimator E(Y(02223)a₁, x)− E(Y∣a₀, x), then bias formulas for d_a+ (x), a_a1 (x), and d_a0 (x) are just like those given in Theorem 1 except the summation of x and the final term in each expression P(x), P(x∣a₁), or P(x∣a₀) would be removed.

We now consider standard errors and confidence intervals for bias-adjusted estimates of the causal effect. First, consider the simple sensitivity-analysis technique with binary U discussed above. Once the sensitivity parameters, corresponding to the relationships between U and A [ie, δ = {P(U = 1∣a₁, x) − P(U = 1∣a₀, x)}]] and between U and Y [ie, γ = {E(Y∣a, x, U = 1) − E(Y∣a, x, U = 0)}]] are fixed, the standard error of the bias-corrected estimator, Σ_x{E(Y∣a₁, x) − E(Y∣a₀, x)}P(x) − δγ, is precisely the same as that of the original estimator. This feature is shared by the results of Lin et al¹¹ discussed in the eAppendix, but not by certain other sensitivity-analysis methods.^{12, 17}

Because the standard errors of the original and bias-adjusted estimates are the same, the bias formulas can be applied not only to the estimate itself but also to both limits of a confidence interval or the original estimate. This useful feature of identical standard errors thus gives rise to a very straightforward method for obtaining confidence intervals for the bias-corrected estimate. These remarks also hold for the bias-corrected estimators of the conditional causal effects discussed above. However, the remarks do not apply to the most general results in Theorem 1 if the sensitivity parameters {E(Y∣a₁, x, u) − E(Y∣a₁, x, u′)},{E(Y∣a₀, x, u) − E(Y∣a₀, x, u′)}, or {P(u∣a₁, x) − P(u∣x)} vary with x; this is because the bias formula for example d_a+ would depend on the distribution of X, which would have to be estimated from the data. When {E(Y∣a₁, x, u) − E(Y∣a₁,x, u′)}, {E(Y∣a₀, x, u) − E(Y∣a₀, x, u′)}, or {P(u∣a₁, x) − P(u∣x)} vary with x, one possible approach would be to obtain bootstrapped samples of the data and to apply the bias formulas to each bootstrapped sample and use the, eg, 2.5th and 97.5th percentiles of the bias-corrected estimates as a 95% confidence interval for the bias-corrected estimate of the original data.²²

Other Measures of Effect

For binary Y, other measures of effect such as the risk ratio or odds ratio may be of interest. The conditional causal risk ratio in the total population or among those receiving treatment a₁ or a₀ are defined respectively by E(Y_a1∣x)/E(Y_a0 ∣x), E(Y_a1 ∣a₁,x)/E(Y_a0 ∣a₁, x), and E(Y_a1 ∣a₀, x)/E(Y_a0 ∣a₀, x). The conditional causal odds ratios in the total population or among those receiving treatment a₁ or a₀ can be defined similarly, eg, the conditional causal odds ratio in the total population is defined by:

$$\frac{E\left(Y_{a_1}\mid x\right)/\left\{1-E\left(Y_{a_1}\mid x\right)\right\}}{E\left(Y_{a_0}\mid x\right)/\left\{1-E\left(Y_{a_0}\mid x\right)\right\}}.$$

We can define bias expressions $d_{a_{+}}^{RR}\left(x\right)$, $d_{a_1}^{RR}\left(x\right)$ and $d_{a_0}^{RR}\left(x\right)$ corresponding to the ratios between the risk ratios conditional on X and the true conditional causal risk ratios respectively in the total population or among those receiving treatment a₁ or a₀:

$$d_{a_{+}}^{RR}\left(x\right)=\frac{E\left(Y\mid a_1,x\right)/E\left(Y\mid a_0,x\right)}{E\left(Y_{a_1}\mid x\right)/E\left(Y_{a_0}\mid x\right)}$$

$$d_{a_1}^{RR}\left(x\right)=\frac{E\left(Y\mid a_1,x\right)/E\left(Y\mid a_0,x\right)}{E\left(Y_{a_1}\mid a_1,x\right)/E\left(Y_{a_0}\mid a_1,x\right)}$$

$$d_{a_0}^{RR}\left(x\right)=\frac{E\left(Y\mid a_1,x\right)/E\left(Y\mid a_0,x\right)}{E\left(Y_{a_1}\mid a_0,x\right)/E\left(Y_{a_0}\mid a_0,x\right)}.$$

We can define the conditional odds-ratio bias term analogously. For the bias terms for the conditional causal risk ratio, we then have the following result, again as with Theorem 1, expressing the biases in terms of the relationship between the unmeasured confounder(s) U and the outcome Y and the relationship between U and treatment group A.

Theorem 2. If Y_a ∐A∣X, U, and if u′ is any chosen reference value for the unmeasured confounder U then

$$d_{a_{+}}^{RR}\left(x\right)=\frac{{\sum }_u\frac{E\left(Y\mid a_1,x,u\right)}{E\left(Y\mid a_1,x,u\prime \right)}P\left(u\mid a_1,x\right)}{{\sum }_u\frac{E\left(Y\mid a_1,x,u\right)}{E\left(Y\mid a_1,x,u\prime \right)}P\left(u\mid x\right)}/\frac{{\sum }_u\frac{E\left(Y\mid a_0,x,u\right)}{E\left(Y\mid a_0,x,u\prime \right)}P\left(u\mid a_0,x\right)}{{\sum }_u\frac{E\left(Y\mid a_0,x,u\right)}{E\left(Y\mid a_0,x,u\prime \right)}P\left(u\mid x\right)}$$

$$d_{a_1}^{RR}(x)=\frac{{\sum }_u\frac{E(Y\mid a_0,x,u)}{E(Y\mid a_0,x,u\prime )}P(u\mid a_1,x)}{{\sum }_u\frac{E(Y\mid a_0,x,u)}{E(Y\mid a_0,x,u\prime )}P(u\mid a_0,x)}$$

$$d_{a_0}^{RR}(x)=\frac{{\sum }_u\frac{E(Y\mid a_1,x,u)}{E(Y\mid a_1,x,u\prime )}P(u\mid a_1,x)}{{\sum }_u\frac{E(Y\mid a_1,x,u)}{E(Y\mid a_1,x,u\prime )}P(u\mid a_0,x)}.$$

If the outcome is rare in all strata of a, x, and u, then the bias formulas given in Theorem 2 will also hold approximately for bias formulas for the conditional causal odds ratio. Exact bias formulas for odds ratios are also given in the eAppendix (http://links.lww.com/EDE/A429) but are somewhat more cumbersome to use. In some cases, we may be interested in marginal or “standardized” causal risk ratios and odds ratios. The marginal or “standardized” causal risk ratio in the total population or among those receiving treatment a₁ or a₀ are defined, respectively, by E(Y_a1)/E(Y_a0), E(Y_a1∣a₁)/E(Y_a0∣a₁), and E(Y_a1∣a₀)/E(Y_a0∣a₀). We can then define the bias expressions $d_{a_{+}}^{RR}$, $d_{a_1}^{RR}$, and $d_{a_0}^{RR}$ for the marginal causal risk ratios by the ratio between the marginalized risk ratios adjusted for X only and the true causal risk ratios:

$$d_{a_{+}}^{RR}=\frac{{\sum }_{x}E(Y\mid a_1,x)P(x)/{\sum }_{x}E(Y\mid a_0,x)P(x)}{E(Y_{a_1})/E(Y_{a_0})}$$

$$d_{a_1}^{RR}(x)=\frac{{\sum }_{x}E(Y\mid a_1,x)P(x\mid a_1)/{\sum }_{x}E(Y\mid a_0,x)P(x\mid a_1)}{E(Y_{a_1}\mid a_1)/E(Y_{a_0}\mid a_1)}$$

$$d_{a_0}^{RR}(x)=\frac{{\sum }_{x}E(Y\mid a_1,x)P(x\mid a_0)/{\sum }_{x}E(Y\mid a_0,x)P(x\mid a_0)}{E(Y_{a_1}\mid a_0)/E(Y_{a_0}\mid a_0)}$$

The marginal or “standardized” causal odds ratio, along with bias term for these marginal causal odds ratios, can be defined similarly. Although the formulas and their derivations are somewhat more complicated, bias results for the marginal causal risk ratio can also be obtained and are given in the following theorem.

Theorem 3. If Y_a∐A∣X, U, and if u′ is any chosen reference value for the unmeasured confounder U then, and if x′ is any chosen reference value for X, we then have the following:

$$d_{a_{+}}^{RR}=\frac{{\sum }_x\frac{E(Y\mid a_1,x)}{E(Y\mid a_1,x\prime )}P(x)}{{\sum }_x{r}_1{(x)}^{-1}\frac{E(Y\mid a_1,x)}{E(Y\mid a_1,x\prime )}P(x)}/\frac{{\sum }_x\frac{E(Y\mid a_0,x)}{E(Y\mid a_0,x\prime )}P(x)}{{\sum }_x{r}_0{(x)}^{-1}\frac{E(Y\mid a_0,x)}{E(Y\mid a_0,x\prime )}P(x)}$$

$$d_{a_1}^{RR}=\frac{{\sum }_x{d}_{a_1}^{RR}(x)\frac{E(Y\mid a_0,x)}{E(Y\mid a_0,x\prime )}P(x\mid a_1)}{{\sum }_x\frac{E(Y\mid a_0,x)}{E(Y\mid a_0,x\prime )}P(x\mid a_1)}$$

$$d_{a_0}^{RR}=\frac{{\sum }_x\frac{E(Y\mid a_1,x)}{E(Y\mid a_1,x\prime )}P(x\mid a_0)}{{\sum }_x{d}_{a_0}^{RR}{(x)}^{-1}\frac{E(Y\mid a_1,x)}{E(Y\mid a_1,x\prime )}P(x\mid a_0)}$$

where,

$$r_1(x)=\frac{{\sum }_u\frac{E\left(Y\mid a_1,x,u\right)}{E\left(Y\mid a_1,x,u\prime \right)}P\left(u\mid a_1,x\right)}{{\sum }_u\frac{E\left(Y\mid a_1,x,u\right)}{E\left(Y\mid a_1,x,u\prime \right)}P(u\mid x)}$$

$$r_0(x)=\frac{{\sum }_u\frac{E\left(Y\mid a_0,x,u\right)}{E\left(Y\mid a_0,x,u\prime \right)}P\left(u\mid a_0,x\right)}{{\sum }_u\frac{E\left(Y\mid a_0,x,u\right)}{E\left(Y\mid a_0,x,u\prime \right)}P(u\mid x)}$$

and where, $d_{a_1}^{RR}(x)$ and $d_{a_0}^{RR}(s)$ are the bias formulas for the conditional risk ratios given in Theorem 2.

Note that in applying Theorem 3, the expressions of the form E(Y∣a, x, u)/E(Y∣a, x, u′) and P(u∣a, x) would be specified in a sensitivity analysis; the expressions of the form E(Y∣a, x)/E(Y∣a, x′) can be estimated from the data. If the outcome is rare in all strata of a, x, and u, then the bias formulas given in Theorem 3 will also hold approximately for bias formulas for the marginal causal odds ratios. As discussed in the eAppendix, these results for the conditional and marginal causal risk ratios and causal odds ratios generalize those for the risk ratios and odds ratios in the external adjustment literature.^7,9,21

We close this section by noting that in the simple sensitivity-analysis setting, in which it is assumed that there is a binary unmeasured confounding variable U such that the relationship between U and Y, ie, $\frac{E(Y\mid a,x,U=1)}{E(Y\mid a,x,U=0)}$, is the same for treatment levels a₁ or a₀ and takes value γ, then^3,4:

$$d_{a_{+}}^{RR}\left(x\right)=\frac{1+\left(\gamma -1\right)P\left(U=1\mid a_1,x\right)}{1+\left(\gamma -1\right)P\left(U=1\mid a_0,x\right)}$$

If the outcome is rare, this simple formula will also hold for $d_{a_{+}}^{OR}(x)$. As with the linear scale, the bias formulas can be applied immediately to the estimate and both limits of a confidence interval in the case of conditional causal risk ratios and odds ratios; however, a bootstrap approach would be necessary for the marginal causal risk ratio or odds ratio.

Applications of Bias Formulas

We illustrate the application of Theorem 1, using 3 examples. The first example highlights the interpretative ease of using a linear scale for sensitivity-analysis parameters (as opposed to, for example, an odds ratio scale); the second and third examples illustrate the generality and flexibility of the sensitivity-analysis technique described above. In the first example, we consider an analysis of the data presented by Rosenbaum and Rubin⁶; in the Appendix we compare and contrast the sensitivity-analysis technique described in this paper to that described in their work. Whereas Rosenbaum and Rubin give the sensitivity-analysis parameters on an odds-ratio scale, the sensitivity-analysis technique described above allows us to specify sensitivity-analysis parameters on a linear scale, which are often easier to interpret.

Rosenbaum and Rubin consider a study comparing coronary artery bypass surgery (A = 1) to medical therapy (A = 0) in the treatment of coronary artery disease. Their outcome, Y, is dichotomous—symptomatic relief after 6 months. Data are available on 74 covariates (X) and they use a propensity-score approach³⁴ to form 5 propensity-score subclasses. The overall adjusted relief rates are 0.67 for surgery and 0.36 for medical therapy, yielding an estimate of the causal effect of 0.31 (95% confidence interval 0.169 to 0.451). In this section, we will apply the formula for d_a+ in Theorem 1 under simplifying assumptions to assess the sensitivity of the estimate of the causal effect to unmeasured confounding. Using the simple sensitivity-analysis technique described above, we see that, if there were a binary unmeasured confounding variable U with a 0.6 higher prevalence among those receiving surgery as compared with medical therapy (in all propensity-score strata), and if the outcome difference comparing those with U = 1 and U = 0 were 5.17 for both treatment groups in all propensity-score strata, then we would obtain a bias term of d_a+ = (0.517)(0.6) = 0.31. Thus such an unmeasured confounding variable would reduce the estimate of the causal effect to 0 (95% CI = −0.141 to 0.141). If, on the other hand, the difference in the prevalence of the unmeasured confounding variable among those receiving surgery as compared with medical therapy were only 0.3 in all propensity score strata, then we would obtain a bias term of d_a+ = (0.517)(0.3) = 0.155 and our estimate of the causal effect would still be 0.31–0.155 = 0.155 (95% CI = 0.014 to 0.296). Admittedly, it seems unlikely that an unmeasured confounding variable could have an effect on the outcome sufficiently large (even after control for 74 covariates) to invalidate the qualitative conclusion of surgery providing higher proportion of relief at 6 months than medical therapy. Further evidence for the conclusion that surgery resulted in greater symptomatic relief than medical therapy is provided by the fact that a 30 percentage point difference in the prevalence of the unmeasured confounder would be inadequate to explain away the estimated effect, even if the unmeasured confounder had an effect of the same magnitude, 0.517.

In our second example, we will consider a study by Reinisch et al³⁵ that examined the effect of in utero exposure to phenobarbital on intelligence in men. Subjects were selected from the largest hospital in Copenhagen. The exposure group consisted of those who had been exposed in utero to phenobarbital (A = 1) and the control group of those who had not (A = 0). Propensity-score matching and regression techniques³⁶ were used to adjust for background characteristics in making intelligence comparisons using the Danish Military Board Intelligence Test (Y) taken by the exposed and unexposed men when they had reached their early 20s. The background characteristics (X) for which adjustment was made included family socioeconomic status, breadwinner's education, sibling position, whether the pregnancy was wanted, whether the mother attempted an abortion, maternal marital status, predisposing risk score indicating conditions were less than optimal for conception, mother's age, father's age, gestational length, birth weight, birth length, number of cigarettes per day in the third trimester, maternal weight gain divided by height cubed, and the maternal complaint score.

Subjects exposed to phenobarbital were found to have significantly lower scores on the Danish Military Board Intelligence Test than they would have had they not been exposed. Specifically, under the assumption that the effect of exposure is unconfounded given X, Reinisch et al obtained an estimate of E(Y₁∣A = 1) of 39.58 and an estimate of E(Y₀∣A = 1) of 44.35 to obtain an estimate of the effect of the exposure on the exposed of E(Y₁∣A = 1) − E(Y₀∣A = 1) = −4.77) (95% CI = −7.96 to −1.58). Reinisch et al suggest that parental intelligence, which was not measured in the study, may partially confound the analysis. Reinisch et al reason informally, without a quantitative analysis, to argue that it is unlikely that parental intelligence, rather than drug effects, are responsible for the observed intelligence deficits. Using Theorem 1 above, if we hypothesize an unmeasured confounding variable U of, say, the average of maternal and paternal intelligence measured by the Danish Military Board Intelligence Test, and we assume that if unexposed, for all x, a one-point increase in U would on average result in a 0.3 point increase in Y so that E(Y∣a₀,x, u) − E(Y∣a₀,x, u′) = (0.3)(u − u′), then it follows from Theorem 1 that d_a1 = (03){E(U∣A = 1) − E(U∣A = 0)}. It would thus require a difference in parental intelligence of E(U∣A = 1) − E(U∣A = 0) = − 4.77/(0.3) = −15.9 between the parents of the exposed and unexposed on the Danish Military Board Intelligence Test to completely eliminate the estimated deficit. Reinisch et al note that the standard deviation for a national sample of subjects taking the Danish Military Board Intelligence Test was 11.38. A 1.3 standard deviation difference in parental intelligence between the exposed and unexposed, although not entirely implausible, seems unlikely.

Note that the sensitivity-analysis technique of Rosenbaum and Rubin⁶ is not applicable to the study of Reinisch et al because the outcome is continuous, not binary; similarly, the prior external-adjustment literature^1,8,21 is not applicable because the outcome is continuous. The sensitivity-analysis techniques of Imbens¹⁴ and Lin et al,¹¹ described in the Appendix, apply to continuous outcomes but presuppose a regression model, whereas Reinisch et al³⁵ obtained results from a matched propensity-score analysis rather than simply through an outcome-regression model. Furthermore, the sensitivity-analysis technique of Lin et al¹¹ presupposes no interaction between the treatment and the covariates, whereas the analysis of Reinisch et al does not assume the absence of such interactions, and, in fact Reinisch et al report evidence of such interactions. Importantly, a sensitivity analysis using the results in Theorem 1 above is not dependent on any particular model or method for obtaining estimates of the prima facie estimate of the causal effect adjusted only for x; sensitivity analysis using Theorem 1 is applicable irrespective of how the initial adjusted estimates are obtained. Furthermore, a sensitivity analysis using the results in Theorem 1 does not presuppose any particular functional form or the absence of interactions.

Our third example illustrates the flexibility of our approach by applying it to a setting in which inverse-probability-of-treatment weighting³⁷ is used to estimate causal effects. This application shows how the bias formulas in Theorem 1 can be employed in greater generality. We use data from the National Center for Health Statistics (NCHS) Birth Certificate Files for year 2000 to consider the effect of adequate prenatal care on birthweight. Prenatal care was classified as adequate versus inadequate, as defined by a modification of the Adequacy of Prenatal Care index.^38,39 Baseline covariates in the NCHS data include maternal age, race, place of birth, place of residence, education, marital status, plurality, gravidity, prior preterm birth, prior birth greater than 4000 g, alcohol consumption, and tobacco use. Socioeconomic status might serve as an unmeasured confounding variable for the relationship between prenatal care and birth weight. Although education may serve as a proxy for socioeconomic status, there are likely aspects of socioeconomic status that education does not capture. The relationships between socioeconomic status and birthweight, and between socioeconomic status and adequate prenatal care, may vary by age. For example, the relationship between socioeconomic status and adequate prenatal care may be weaker for those age 19 years or under at the time of childbirth than for those at older ages, due to State Children's Health Insurance Programs (SCHIP). On the other hand, the birthweight of infants of younger mothers may be more sensitive to adverse socioeconomic circumstances. Inverse probability of treatment weighting³⁷ was used to obtain estimates of the effect of prenatal care on birthweight, stratified by age (≤19 vs. >19 years), controlling by weighting for the aforementioned confounding variables. The estimated effect for younger mothers was 82.4 g (95% CI = 78.2 to 86.7) and for older mothers was 78.3 g (95% CI = 76.3 to 80.3); 12.2% of the mothers were age 19 years or under, and thus the overall estimate of the effect is 78.8 g (95% CI = 76.4 to 81.2). Let U denote adverse versus adequate socioeconomic status. Suppose we hypothesize that the average effect of adverse socioeconomic status for younger mothers is 120 g and the average effect for older mothers is 80 g whereas the difference in the likelihood of adverse socioeconomic status, comparing those with adequate versus inadequate prenatal care, was only 20% for young mothers (due to SCHIP); however, for older mothers, comparing those receiving adequate versus inadequate prenatal care, the difference was 50%. Employing the bias formula in Theorem 1, we would have a corrected estimate of effect of 40.8 g (95% CI = 38.3 to 43.2). Other values for the sensitivity-analysis parameters could similarly be considered. For example, if the effect sizes for the unmeasured confounder were doubled to 240 and 160 g for younger and older mothers, respectively, this would reverse the sign of the point estimate of the effect. Here, in contrast to the first 2 examples, the values of the sensitivity-analysis parameters needed to completely explain away the effect are perhaps not as implausible.

In this third example, we have allowed the effect of the sensitivity-analysis parameters to vary over one covariate (age), and we thus obtain an approach between the complexity of the simple technique described above and the fully general formula given in Theorem 1. Of course it would also be possible to allow the sensitivity parameter to vary across other measured covariates; however, as noted in the discussion following Theorem 1, the more general the approach, the more sensitivity-analysis parameters need to be specified.

In the eAppendix (http://links.lww.com/EDE/A429), to yet further demonstrate the flexibility of our approach, we apply it to a recent study⁴⁰ employing a doubly robust estimator for treatment effects.^41–44

Discussion

In this paper, we have derived bias formulas for sensitivity analysis for causal effects. These formulas have allowed for binary, ordinal, or continuous outcomes; categorical or continuous treatment; and categorical or continuous measured and unmeasured confounding variables. We have obtained results for additive, risk-ratio and odds-ratio scales. We have shown that these bias formulas generalize many of the existing sensitivity-analysis results in the bias-modeling literature, and can be used in a broad range of settings; the results do not presuppose a particular functional form relating the outcome and the observed covariates and treatment. Furthermore, the results can be used to perform a very simple form of sensitivity analysis. The sensitivity-analysis approach that we have taken fixes the sensitivity parameters and considers how the conclusions would be affected; an alternative Bayesian approach gives a distribution of parameter values and incorporates the uncertainty of this distribution into the confidence intervals for the corrected estimate.^15,16,19

As noted above, a researcher can hypothesize a binary unmeasured confounding variable U with constant effect γ on the outcome across treatment and covariate levels and with a constant prevalence difference, δ, comparing treatment levels a₁ and a₀ across strata of the covariates. Under these simplifying assumptions, the bias comparing the estimated outcome difference (adjusted only for measured covariates X) and the true causal effect (adjusted for both X and U) is then given simply by δγ. We have shown that this simple technique is in fact applicable irrespective of the particular method by which the initial adjusted estimates of the causal effects were obtained, and furthermore does not assume the absence of interactions between treatment and covariates. Although, in many contexts, more sophisticated sensitivity-analysis techniques may be desirable, this simple approach leaves researchers without excuse for not performing, at the very least, a simple sensitivity analysis.

Appendix

Proof of Theorem 1

We give the proof for d_a+. The proofs for d_a1 and d_a0 are given in the eAppendix (http://links.lww.com/EDE/A429).

$$d_{a+}=\{{\sum }_{x}E(Y\mid a_1,x)P(x)-{\sum }_{x}E(Y\mid a_0,x)P(x)\}-\{E(Y_{a1})-E(Y_{a0})\}={\sum }_x{\sum }_{u}E(Y\mid a_1,x,u)P(u\mid a_1,x)P(x)-{\sum }_x{\sum }_{u}E(Y\mid a_0,x,u)P(u\mid a_0,x)P(x)-{\sum }_x{\sum }_{u}E(Y_{a1}\mid x,u)P(u\mid x)P(x)+{\sum }_x{\sum }_{u}E(Y_{a0}\mid x,u)P(u\mid x)P(x)$$

by iterated expectations

$$={\sum }_x{\sum }_{u}E(Y\mid a_{1,}x,u)P(u\mid a_1,x)P(x)-{\sum }_x{\sum }_{u}E(Y\mid a_0,x,u)P(u\mid a_0,x)P(x)-{\sum }_x{\sum }_{u}E(Y_{a1}\mid a_1,x,u)P(u\mid x)P(x)+{\sum }_x{\sum }_{u}E(Y_{a_0}\mid a_0,x,u)P(u\mid x)P(x)$$

by unconfoundedness conditional on (U,X)

$$={\sum }_x{\sum }_{u}E(Y\mid a_1,x,u)\{P(u\mid a_1,x)-P(u\mid x)\}P(x)-{\sum }_x{\sum }_{u}E(Y\mid a_0,x,u)\{P(u\mid a_0,x)-P(u\mid x)\}P(x)$$

by consistency

$$={\sum }_x{\sum }_u\{E(Y\mid a_1,x,u)-E(Y\mid a_1,x,u^{\prime })\}\{P(u\mid a_1,x)-P(u\mid x)\}P(x)-{\sum }_x{\sum }_u\{E(Y\mid a_0,x,u)-E(Y\mid a_0,x,u^{\prime })\}\{P(u\mid a_0,x)-P(u\mid x)\}P(x)$$

since E(Y∣a₁, x, u′) and E(Y∣a₁, x, u′) are constants.

Proof of Theorems 2 and 3

See eAppendix (http://links.lww.com/EDE/A429) for details.

Relation to Other Sensitivity Analysis Techniques

Relation to the Sensitivity Analysis of Rosenbaum and Rubin (1983)

In the sensitivity analysis proposed by Rosenbaum and Rubin,⁶ they consider a binary outcome Y, binary treatment A, covariate(s) X (in their application, X indicates propensity score strata), and hypothesize a binary unmeasured confounder U such that Y_a∐A∣X, U. The researcher specifies sensitivity parameters

$${\pi }_x=P(U=1\mid x)$$

$${\alpha }_x=log\left(\frac{P(A=0\mid U=1,X=x)}{1-P(A=0\mid U=1,X=x)}/\frac{P(A=0\mid U=0,X=x)}{1-P(A=0\mid U=1,X=x)}\right)$$

$${\delta }_{xt}=log\left(\frac{P(Y=0\mid U=1,A=t,X=x)}{1-P(Y=0\mid U=1,A=t,X=x)}/\frac{P(Y=0\mid U=0,A=t,X=x)}{1-P\left(Y=0\mid U=0,A=t,X=x\right)}\right).$$

For specified values of π_x, α_x, and δ_xt, maximum likelihood estimates of the causal effect can then be obtained; sensitivity analysis proceeds by specifying different values of the parameters π_x,α_x, and δ_xt. In their application (described above), π_x,α_x, and δ_xt are assumed to be constant over x. As noted above, assuming no unmeasured confounding the estimate of the causal effect is 0.31. Rosenbaum and Rubin⁶ consider values for π (the overall prevalence of U) of 0.1, 0.5, and 0.9. They first consider α = 2 and values δ_t=0 and δ_{t= 1} either $\frac{1}{2}$ or 2; under this set of scenarios, the smallest causal effect estimate, an estimate of 0.28, is obtained when the prevalence of U is 0.5 and when U doubles the odds of surgery (α = 2) and also doubles the odds of improvement (δ_t=0 = 2 and δ_{t= 1} = 2). They then consider α = 3 and values δ_t=0 and δ_{t= 1} either $\frac{1}{3}$ or 3; under this set of scenarios, the smallest causal effect estimate, an estimate of 0.25, is obtained when the prevalence of U is 0.5 and when U triples the odds of surgery (α = 3) and also triples the odds of improvement (δ_t=0 = 3 and δ_{t= 1} = 3). They conclude that for an unobserved confounder to explain the outcome difference comparing medical and surgical patients, it would have to more than triple the odds of surgery and of improvement. Admittedly, this seems unlikely. We saw above, however, that when the sensitivity analysis is conducted on a risk-difference scale rather than an odds-ratio scale, although the degree of uncontrolled confounding that would be needed to explain away the estimate of the causal effect is still unlikely, the numbers are perhaps slightly less inconceivable.

The bias formula for d_a+ in Theorem 1 can also be used in a reasonably straightforward way to replicate the odds-ratio sensitivity analysis of Rosenbaum and Rubin.⁶ Let α₀ = 0 and α₁ = 1 and let u = 1 and u′ = 0. For specified values of the sensitivity-analysis parameters π_x,α_x, and δ_xt in the approach proposed by Rosenbaum and Rubin we will show how to obtain the quantities needed for the application of the formula for d_a+ in Theorem 1, namely, (i) {E(Y∣a₁, x, u) − E(Y∣a₁, x, u′)} and {E(Y∣a₀, x, u} − E(Y∣a₀, x, u′)} and (ii) {P(u∣a₁, x) − P(u∣x)} and {P(u/a₀, x) − P(u ∣x)}. Given α_x and π_x, we can use the equations

$${\alpha }_x=log\left(\frac{P(a_0\mid x,u)}{1-P(a_0\mid x,u)}/\frac{P(a_0\mid x,u^{\prime })}{1-P(a_0\mid x,u^{\prime })}\right)$$

$$P(a_0\mid x)=P(a_0\mid x,u)P(u\mid x)+P(a_0\mid x,u^{\prime })P\left(u^{\prime }\mid x\right)$$

to solve for P(a₀∣x, u) and P(a₀∣x, u′); note that P(u∣x) = π_x and P(u′∣x) = 1 − π_x. It then follows from Bayes' rule that

$$P(u\mid a_0,x)=\frac{P(a_0\mid x,u){\pi }_x}{P(a_0\mid x)}.$$

A similar procedure can be used to obtain P(u ∣ a₁, x) and thus also P(u∣x). Furthermore, to obtain {E(Y∣a₁,x, u) − E(Y∣a₁,x, u′)} and {E(Y∣a₀,x, u) − E(Y∣α₀, x, u′)}, we can use the equations

$${\delta }_{x0}=log\left(\frac{1-E(Y\mid a_0,x,u)}{E(Y\mid a_0,x,u)}/\frac{1-E(Y\mid a_0,x,u^{\prime })}{E(Y\mid a_0,x,u^{\prime })}\right)$$

$$E(Y,U\mid a_0,x)=E(Y\mid a_0,x,u)P(u\mid a_0,x)+E(Y\mid a_0,x,u^{\prime })P(u^{\prime }\mid a_0,x)$$

to solve for E(Y∣α₀,x,u) and E(Y∣α₀, x, u′); note that P(u∣a₀, x) and P(u′∣α₀, x) have already been obtained. A similar procedure can be used to obtain E(Y∣α₁,x, u) and E(Y∣ α₁, x, u′). We can then proceed by using the bias formula for d_a+.

Although the bias formula for d_a+ in Theorem 1 can be used to replicate the odds-ratio sensitivity analysis of Rosenbaum and Rubin,⁶ the formula in Theorem 1 is considerably more general since, as we have seen in the applications above, it can be applied to binary or continuous outcomes and to binary, categorical, or continuous confounding variables and treatment variables. The result of Rosenbaum and Rubin was restricted to binary outcomes.

Relation to Sensitivity Analysis of Lin et al (1998)

Lin et al¹¹ considered settings including binary and continuous outcomes Y, binary treatment A, and binary and continuous unmeasured confounding variable U. They compared the 2 regression models

$$E(Y\mid a,u,x)=g(\alpha +\beta a+\gamma u+{\theta }^{\prime }x)$$

and

$$E\left(Y\mid a,x\right)=g\left({\alpha }^{\ast }+{\beta }^{\ast }a+{\theta }^{\ast }\prime x\right).$$

for linear, log-linear, and logistic links g and derived algebraic formulas to relate β and β* under 2 possible alternative assumptions. Their first assumption was that U and X were conditionally independent given A. Their second possible assumption was that the mean of U conditional on A and X was additive in A and X, ie, μ_a,x: = E(U∣ A = a, X = x) = μ_α + q(x). Hernán and Robins⁴⁵ showed that the first assumption concerning the independence of U and X conditional on A could not be satisfied if both U and X were causes of A, and thus that the results of Lin et al¹¹ concerning the conditional independence assumption could not be employed in those contexts in which the formulas would be most useful, ie, when both U and X contained confounding variables. VanderWeele²⁰ showed that the second assumption of Lin et al concerning additivity held for an entire family of distributions even if both U and X were causes of A. Under this second assumption of additivity, Lin et al¹¹ showed that when the conditional distribution of U given X and A is normal with mean μ_α,x = μ_α and q(x) then the regression coefficients β and β* were related by

$$\beta ={\beta }^{\ast }-\gamma ({\mu }_1-{\mu }_0)$$

for linear and log-linear links and that this relationship held approximately for a logistic link. Lin et al¹¹ also noted that this relationship would hold for linear link (but not log-linear or logistic) if U were binary (rather than normally distributed) with a conditional mean μ_α,x = μ_α and q(x).

The results for a linear link follow immediately from the bias formula for d_a+ (x) by replacing expressions of the form E(Y∣ a, u, x) with the linear combination of regression coefficients and by using E(U∣a, x) = μ_a + q(x). See also Cox and McCullagh.³² Furthermore, from the bias formula for d_a+ it also follows that to obtain β = β* − γ(μ₁ − μ₀), the unmeasured confounding variable U does not need to be binary nor to be conditionally normally distribution; all that is needed to obtain β = β* − γ(μ₁ − μ₀), or more generally to obtain β = β* − γ(μ_a1 − μ_a0), is that the conditional mean of U given X and A is given by μ_a,x = μ_a + q(x). Not only do all of these results for linear regression follow immediately from Theorem 1, but the bias formula for d_a+ (x) can in fact be used to relax the additivity assumption. Without making an assumption about additivity, it follows immediately from the bias formula for d_a+ that β and β* are related by

$$\beta ={\beta }^{\ast }-\gamma {\int }_x{\int }_u\{dP(u\mid a_1,x)-dP(u\mid a_0,x)\}dP(x).$$

The bias formulas in Theorem 1 are yet more general than this in that, as we saw in the applications above, unlike the results of Lin et al¹¹ and Imbens,¹⁴ Theorem 1 does not presuppose a regression context; furthermore it does not assume that there are no interactions between A, U, and X, and does not presuppose any particular functional form.

Relation to Prior External Adjustment Results

As noted above, the bias formulas in Theorem 1 are a generalization of prior bias formulas in the external adjustment literature.¹,⁸,²¹ This prior external adjustment literature generally considered the setting of a dichotomous treatment, a dichotomous outcome, and a categorical unmeasured covariate. If in Theorem 1, X = Ø, Y is dichotomous and U is categorical, the formulas given above reduce to the results for the risk difference in Kitagawa¹ and Arah et al.²¹ The bias formula results given here thus generalize these prior external adjustment results in 2 ways. First, our results apply not only to dichotomous outcomes but also to continuous outcomes. Second, our results allow for control of some set of measured covariates X. Whereas prior risk difference results compared the average outcome difference unadjusted for X, E(Y∣a₁) − E(Y∣a₀), to the causal effect E(Y_a1) − E(Y_a0), our results compare the average outcome difference adjusted for X, Σ_x{E(Y∣a₁, x) − E(Y∣a₀, x)}P(x), to the causal effect E(Y_a1) − E(Y_a0). Most of the prior external adjustment literature effectively presupposed that the analysis was within strata of, or conditional on, X (or that there were no measured covariates to control for). To combine results over strata of X, Lee and Wang⁴⁶ and Flanders and Khoury⁹ for instance propose an assumption of homogeneity of effects across of X; Greenland¹⁵,¹⁶ and Arah et al²¹ discuss Bayesian and Monte Carlo method approaches; using risk ratios, Flanders and Khoury⁹ also derive an external adjustment formula in cases with both a measured and an unmeasured covariate.