Mathur and Shpitser respond to “The evolution of selection bias in the recent epidemiologic literature—a selective overview”

We thank Lu et al¹ for their response to our article on graphical rules for selection bias.² They provide an insightful and interesting overview of methodological developments regarding selection bias in epidemiology. We reflect on certain topics they address and on areas of active research, drawing on research in epidemiology and adjacent disciplines.

Lu et al¹ emphasize the need to consider the practical utility of estimands that may be used in the presence of selection bias, such as the net treatment difference (described in our article²). We certainly agree. Other estimands have been proposed, and identification conditions proven, for situations that are analogous to selection bias. For example, in the context of survival analysis, the survivor average causal effect (SACE) is the effect of the treatment on the outcome for the subset of individuals who would have survived regardless of treatment status.³ In the context of treatment-affected selection, the SACE would represent the effect of treating the subset of individuals who would be in the selected group regardless of treatment. With treatment-affected selection, it can also be informative to consider the effects of the treatment while simultaneously intervening to set the selection indicator to 1 (eg, to prevent missing data from occurring).^2,4,5 We provided some results for this case.² Each of these estimands requires different identification assumptions, whose plausibility will depend on scientific context.^2,3

To date in epidemiology, discussion of selection bias has largely focused on directed acyclic graphs (DAGs) containing a single selection indicator. That is, each participant is either selected into analysis, in which case all of their variables are observed, or else is not selected, in which case none of their variables is observed. However, in the context of selection due to missing data, some participants may have incomplete data. Graphical models have been developed to accommodate separate indicators for each missing variable.^4-6 Such approaches have yielded interesting and simple identification results. For example, if one wants to adjust for systematic selection using popular approaches such as multiple imputation or inverse-probability weighting, this is possible if and only if the probability of being a complete case given only the observed part of the data is itself identifiable.⁴ In addition, the form of the identifying functional for the complete-case probability will in turn influence the form the inverse weights or imputing distributions must take.

If the joint distribution of the analysis variables is indeed identified, then any estimand pertaining to those variables—including causal estimands such as average treatment effects, but also noncausal estimands such as simple means and proportions—is also identified. Graphical models for missing data remain an active area of research, with recent work considering methods for missing data in the presence of dependent data, such as interference settings.⁷

Systematic selection is a recurring challenge in data analysis and subsumes a number of seemingly disparate areas of research. In particular, transportability/generalizability problems and data fusion problems both feature multiple domains where units may be selected into a particular domain systematically. In transportability/generalizability problems, the aim is to apply findings in one context to another, with appropriate adjustments. In data fusion problems, multiple data sources are combined to address a single question. In addition, classical causal inference in nonrandomized studies and missing data problems feature a systematic selection of the treatment or of a selection indicator. Finally, outcome-dependent sampling and principled analysis of case-control studies must involve appropriate adjustment for “inconvenient” selection.

Causal inference is often viewed as a missing data problem.⁸ On the other hand, causal inference ideas have similarly been used to derive new identification theory in missing data settings.^4-6 Similarly, the conceptual similarity of the wealth of problems that feature systematic selection has yet to be fully explored. We look forward to further developments along these lines.

Acknowledgments

M.B.M. and I.S. co-wrote the manuscript.

Funding

M.B.M. was supported by National Institutes of Health grants R01 LM013866, UL1TR003142, P30CA124435, and P30DK116074. I.S. was supported by ONR N00014-21-1-2820, NSF 2040804, NSF CAREER 1942239 and NIH R01 AI127271-01A1. The funders had no role in the design, conduct, or reporting of this work.

Conflict of interest

The authors declare no conflicts of interest.