Exploring the subtleties of inverse probability weighting and marginal structural models

Abstract

Since being introduced to epidemiology in 2000, marginal structural models have become a commonly used method for causal inference in a wide range of epidemiologic settings. In this brief report, we aim to explore three subtleties of marginal structural models. First, we distinguish marginal structural models from the inverse probability weighting estimator, and we emphasize that marginal structural models are not only for longitudinal exposures. Second, we explore the meaning of the word ‘marginal’ in ‘marginal structural model.’ Lastly, we show that the specification of a marginal structural model can have important implications for the interpretation of its parameters. Each of these concepts have important implications for the use and understanding of marginal structural models, and thus providing detailed explanations of them may lead to better practices for the field of epidemiology.

Introduction

Marginal structural models (MSM), and their estimation using inverse probability weighted (IPW) estimating equations, were introduced by Robins in 1997¹, and to the epidemiology community in two papers published in 2000.^2,3 MSMs have become common in the epidemiologic literature, and have been applied to a range of subjects, from infectious diseases⁴ to maternal and child health⁵ to social⁶ epidemiology.

This brief report aims to explore three subtleties of MSMs. We assume the reader is familiar with IPW approaches (if not we recommend Sato and Matsuyama 2003⁷, Cole and Hernán 2004⁸, and Cole and Hernán 2008⁹) and MSMs (we recommend Robins et al. 2000², Hernán et al. 2000³, Petersen et al. 2006¹⁰, and Hernán et al. 2008¹¹).

Marginal structural models are distinct from inverse probability weighting

The form of a MSM is E(Y^ā) = f(ā; θ), where ā = (a₀, a₁, …, a_T) is the full exposure history, Y^ā is the outcome that the subject would have experienced under exposure history ā, and θ is a set of parameters.² While often considered a tool for longitudinal, time-varying exposures, MSMs can be used in point-exposure settings. For example, we might use the model E(Y^a) = α + θa for dichotomous a and continuous Y; and θ would be interpreted as the average treatment effect¹² of a on Y.

The parameter θ can be estimated in several ways, one of which is IPW estimating equations. The distinction between MSMs and IPW lies in the difference between a parameter (also referred to as an estimand –the thing we are trying to estimate – in this case, θ ) and an estimator (the method or algorithm we are using to estimate it). Estimands may further be categorized as causal, in which case they may be interpreted as the effect of an intervention, and purely statistical, in which case it is a parameter or feature of a distribution that may lack a causal interpretation. IPW is an estimator, but not the only estimator, for the causal estimand (the parameter of the MSM). The distinction between IPW and MSMs (and estimators and estimands more generally) is important, as other methods besides IPW can be used to estimate MSM parameters, such as maximum likelihood^10,13 or targeted maximum likelihood estimation¹⁴.

The ‘marginal’ of ‘marginal structural models’ is not the ‘marginal’ of standardized estimates

Typically in statistical and epidemiologic literature, a model is considered marginal if it is not conditional on covariates, otherwise it is conditional¹⁵. A regression model that includes covariates in the linear predictor to control for confounding is a conditional model. In contrast, standardization methods also control for confounding, but produce marginal estimates with respect to the covariates. Because IPW is a standardization method, it is easy to assume that the marginal in MSM signifies “unconditional on covariates”; this is often true but not the original intended meaning. In the first paper to introduce MSMs to epidemiologists, Robins et al. state “[MSMs] are marginal models, because they model the marginal distribution of the counterfactual random variables Y¹ and Y⁰ rather than the joint distribution.”² In other words, MSMs are marginal because they model the marginal distribution of the potential outcomes. The Appendix explores this distinction in more detail.

Further, when conditional causal effects are of interest or more precise parameter estimation is desired (through the use of stabilized weights), MSMs can be conditional on baseline covariates, yet are still referred to as MSMs because they model the marginal distributions of the potential outcomes; indeed, this is what was done in the companion paper (Hernán et al. 2000³) to the Robins et al. introduction excerpted above. Moreover, history-adjusted MSMs can condition on time-varying confounders.¹⁶ Understanding this distinction is important for interpreting and implementing MSMs, for instance by helping researchers recognize that their MSMs can be conditional models, or that the estimated parameters may not represent average causal effects for their entire study population.

MSM misspecification can lead to the right answer to the wrong question

So-called “causal inference methods” are often used to analyze observational studies like randomized trials. In particular, MSM parameters are often estimated for studying effects analogous to a per-protocol effect from a trial.¹¹ To interpret the estimates from a MSM as emulating such an analysis it is necessary to properly specify the structural model.

Consider the causal diagram in the Figure. This diagram represents a study in which participants are randomized to initiate treatment at time 1 (A₁), and, depending on response to treatment (or lack of treatment) (Z), decide whether or not to stay on treatment (or initiate treatment) at time 2 (A₂). Subjects can experience the outcome of interest at the end of time 1 (Y₁) or the end of time 2 (Y₂). Additionally, the effect of treatment is delayed, so time 1 treatment only effects the outcome at time 2, and treatment only operates through the treatment response, Z. For simplicity, we will assume that all variables are dichotomous.

Figure.

The causal diagram used in the example

The probability of the outcome by time 2 under each potential exposure history can be modeled with an MSM of the form

$$P\left(Y_2^{\overline{a}}=1\right)=1-\prod _{t=1}^2\left(1-h_t^{\overline{a}}\right)$$

where $h_t^{\overline{a}}=P(Y_t^{\overline{a}}=1\mid Y_{t-1}^{\overline{a}}=0)$ is the discrete-time hazard for the outcome at time t (the risk of experiencing the outcome at time t conditional on not experiencing the outcome by t − 1) under exposure regime ā. The MSM specification thus only requires specifying a model for the discrete-time hazard. A non-parametric MSM for the discrete-time hazard is

$$h_t^{\overline{a}}=\{\begin{array}{rr}\hfill {\alpha }_1+{\beta }_1{a}_1,& \hfill t=1\\ \hfill {\alpha }_2+{\beta }_2{a}_1+{\beta }_3{a}_2+{\beta }_4{a}_1{a}_2,& \hfill t=2\end{array}$$ (1)

If such a model is used and the weights are properly estimated, the parameters of this MSM can be validly estimated using IPW, and the results of the analysis will mimic the results of a trial in which subjects are randomized to exposure regimens at baseline.

The MSMs often fit, such as the marginal structural Cox models in Hernán et al 2000³ and Cole et al 2003⁴, and the IPW survival curves in Westreich et al 2010¹⁷, are somewhat different. The MSM above accounts for the full history of exposure, whereas the MSMs used in the papers mentioned here only account for the most recent exposure. An example of a parametric MSM for the discrete-time hazard that only accounts for the most recent exposure is

$${h^{\prime }}_t(\overline{a})=\{\begin{array}{ll}{\alpha }_1+{\beta }_1{a}_1,\hfill & t=1\hfill \\ {\alpha }_2+{\beta }_2{a}_2,\hfill & t=2\hfill \end{array}$$ (2)

Unfortunately, $h_t^{\overline{a}}$ and h′_t(ā) are not necessarily equal. If exposures besides the most recent impact the outcome, then h′_t(ā) may be misspecified and may not represent the true discrete-time hazard at time t under exposure history ā. If only the most recent exposure has an effect on the outcome, then $h_t^{\overline{a}}$ and h′_t(ā) will agree. Of note, when the MSM is misspecified, the use of stabilized weights versus unstabilized weights can change the parameter being estimated.¹⁸

Consider the scenario described in the Table. In this case, the results obtained from estimating (1) with unstabilized IPW (in an infinitely large population with non-parametrically estimated weights) give a causal risk difference for always versus never exposed of 0.44, whereas the results obtained from estimating (2) with IPW under similar conditions give 0.23 (details of these calculations are described in the eAppendix). The risk difference of 0.44 can be interpreted as the difference in the risk of the outcome if all subjects had been exposed in both periods compared with if they had been unexposed in both periods. However, the risk difference of 0.23 is interpreted differently, as described in Westreich et al 2010¹⁷, as the result of a trial in which participants are randomly assigned exposure in the first period and are then re-randomized to exposure in the second period. As described in Cole et al 2012¹⁹, this risk difference could equivalently be interpreted as the result from a trial in which participants are randomized at baseline to begin exposure in the first period, second period, or not at all. It is therefore incorrect to interpret the risk difference of 0.23 as a comparison of the outcome among always treated versus never treated subjects. One suggested way to reduce the bias when using (2) is to change the time scale to time on treatment.¹⁷

Table.

The dependency structure used in the illustrative example

P(A1 = 1) = 0.5

P(Y1 = 1|A1 = a1) = 0.1 + 0.1a1

P(Z = 1|A1 = a1, Y1 = 0) = 0.1 + 0.4a1

P(A2 = 1|Z = z, A1 = a1, Y1 = 0) = 0.2 + 0.6z

P(Y2 = 1|A2 = a2, A1 = a1, Z = z, Y1 = 0) = 0.1 + 0.2a2 + 0.6z

This example highlights an important concept: proper specification of the MSM can dictate the interpretation of the results. As demonstrated here, with a misspecified MSM the results may still have a causal interpretation, but they may not necessarily represent any meaningful or interesting quantity and may be different than the quantity the investigator seeks to estimate.

Discussion

In this brief report, we highlighted and explained three concepts about MSMs and IPW. Though these ideas may seem subtle, they can have important implications for the use and interpretation of MSMs and IPW. First, it is important to know that estimators besides IPW can be used to estimate the parameters of MSMs. Second, it is important to distinguish between estimates that are conditional on covariates versus those that are marginal, and not to assume that because the word “marginal” is in the name marginal structural model that the estimate is marginal in all senses. Lastly, researchers must carefully specify their MSM and ensure that they properly interpret their results. We hope that by elucidating these issues, epidemiologists will gain a deeper understanding into these important methods for causal inference.

Appendix: The meaning of marginal in marginal structural models

Here we elaborate on what it means for a model to be marginal with respect to the potential outcomes. Consider table A1:

Table A1.

Illustration of ‘marginal’ for marginal structural models.

	Y1=1	Y1=0	Margin of Y0
Y0=1	Pr(Y0=1, Y1=1)	Pr(Y0=1, Y1=0)	Pr(Y0=1)
Y0=0	Pr(Y0=0, Y1=1)	Pr(Y0=0, Y1=0)	Pr(Y0=0)
Margin of Y1	Pr(Y1=1)	Pr(Y1=0)

The rows represent the potential outcome for an individual had they not been exposed, and the columns represent the potential outcomes for an individual had they been exposed. Note that the four interior cells of the table correspond to the proportions of the four causal types described by Greenland and Robins²⁰, specifically subjects in the upper left are “doomed” because they experience the outcome regardless of the value of X (Y⁰=Y¹=1), those in the upper right are “helped” because without the intervention they experience the outcome (Y⁰=1) but with the intervention they do not experience the outcome (Y¹=0), those in the bottom left are “harmed,” and those in the bottom right are “immune.” The margins on the right and on bottom are the proportion of the population that would have each level of the outcome if they had been unexposed and exposed, respectively: for example, the margin Pr(Y¹=1) is the proportion of the population that would experience the outcome if, possibly counter to fact, they had been exposed, regardless of what they would experience if they had been unexposed.

Because only one of the potential outcomes for any given individual can be observed, it is difficult to see how one could estimate any of the quantities from the interior cells of the table. Intuitively, this means that we cannot determine how many individuals are doomed, helped, harmed, or immune without further (unverifiable) assumptions. However, under a sufficient set of identification conditions, which might include conditional exchangeability, positivity, consistency, and no exposure or outcome measurement error^21,22, it is possible to estimate the quantities on the margins of the table using observed data (including data observed in a randomized trial), namely the proportion of individuals who would experience each level of the outcome if they had been exposed, and the proportion who would experience each level of the outcome if they had been unexposed. The fact that MSMs estimate a contrast in the latter quantities, which correspond to the margins of the distribution, is what is meant by the word marginal in this context.²