Toward Enhancing the Rigor of Causal-Inference Studies

To the Editor:

In an apparently unprecedented and praiseworthy move (1), 47 editors of 35 medical journals recently published a guidance document on “the design and reporting of observational causal inference studies” (2). The eminent group of authors was motivated by a “call for increased rigor in observational research methods,” but unfortunately the rigor-promoting document itself contains some confusing or untenable ideas, in my opinion.

First, two of the “questions about etiology” provided by the authors for illustrating why they “use causal inference” are actually not about etiology, with the misrepresentation of this concept apparently stemming from the failure to distinguish the etiologic versus interventive genres of causality in medicine (3). Another consequence of this failure is the authors’ encouragement to design observational studies by emulating clinical trials, even if these trials represent an ill-chosen paradigm for etiologic research (3).

Second, the authors “make a distinction between causal inference and prediction modeling,” but in medicine, thinking of a person’s future course of health requires consideration of potential adoption of particular interventions and/or lifestyles that might be reasonable to consider in any given situation—and ipso facto, it requires consideration of the magnitude of anticipated effects of these actions (4). Thus, the topic at issue here is both “predictive” (i.e., prognostic) and causal, so when assessing, for example, a person’s risk of developing lung cancer, it would be important to consider the person’s anticipated cigarette-smoking–related behavior, as the risk depends—causally—on it (5). And another upshot of this is that clinical trials (and their nonexperimental counterparts) should be seen as causal-prognostic studies.

Third, according to the authors’ description of the “historical approach to defining a confounder,” a confounder must be “a cause of the outcome of interest,” whereas the association of the confounder with the exposure at issue is strictly a matter of “prior knowledge.” However, I believe that in the “historical,” pre-directed-acyclic-graph–era conception, a confounder can be a noncausal determinant of the outcome’s occurrence, whereas the exposure–confounder association is viewed as an ad hoc, study-base–specific matter.

Fourth, in reference to the scenario depicted in Figure 1C (2), the authors state that “controlling for a collider will open the back-door path, thereby introducing confounding,” but I do not believe this view accords with the proper conception of confounding (whether it be considered from the historical or modern vantage.)

Fifth, according to the authors, “it is reasonable to use the label ‘causal association’…to describe findings arising from an observational study,” but a causal association is not something that can be found and described in a study, as causality is unobservable.

Sixth, I found it puzzling that in reference to the example illustrating their recommended way of reporting study results (2, p. 27), the authors characterize the “effect estimate” as “imprecise,” while characterizing the “point and interval estimates” of the effect as “informative” (with this apparent incoherence highlighting the fact that “precise vs. imprecise”—like “informative vs. noninformative” and “significant vs. nonsignificant”—is an arbitrary and subjective dichotomy.) Furthermore, the authors’ interpretation of the upper 95% confidence limit for that rate ratio estimate as meaning that “a rate ratio as large as 4.2 has not been plausibly excluded” incorrectly implies that rate ratio values greater than 4.2 have been “plausibly excluded” (6).

Finally, the authors recommend “interpreting the variability around an effect estimate when making conclusions about causal associations,” implying that results of a single study can be conclusive regarding causality, but this can hardly ever happen, especially in an observational study. Thus, “making conclusions about causal associations” by any given study’s author(s) should be proscribed (and accordingly, the Conclusion(s) section in articles in medical journals should be abolished).