Causal Inference and Ventilator-associated Pneumonia-attributable Mortality in COVID-19: On Painting Bull’s-Eyes and Dotting i’s

To the Editor:

In a recent issue of the Journal, Vacheron and colleagues (1) found that ventilator-associated pneumonia (VAP) after intubation is more common (32 vs. 14–18 episodes per 1,000 ventilation days) and more harmful (about 9.2% vs. 1.2–1.4% of deaths attributable to VAP) among patients with coronavirus disease (COVID-19). As pointed out, the (population) attributable fraction (PAF) is determined by the efficacy of both preventive and therapeutic efforts. It, therefore, captures not only excess mortality (despite therapeutic efforts) but also the elevated VAP incidence. To isolate the former component, they additionally estimated the “attributable mortality” (2).

We commend the authors for being explicit about these targeted effect measures (see their Table 1) and for highlighting their subtle but often overlooked differences. Being clear and explicit about what is being estimated (the “estimand”) fosters meaningful interpretations and helps to avoid semantic confusion. Yet, we are concerned that despite this level of transparency, they may (unintendedly) contribute to pervasive misinterpretations of attributable mortality measures, which are often (if not always) intended to be interpreted causally. For instance, they define the PAF as “the proportion of deaths that would not have occurred if no patient presented the exposure at the time (t)” (p. 163), explicitly referring to a hypothetical scenario in which everyone had—possibly counter to the fact—remained unexposed to VAP. This deceivingly raises the impression that they target a causal estimand that compares the risk (of death) in the same population under this counterfactual scenario with the factually observed scenario in which a fraction of patients acquire VAP. Given its hypothetical nature, the counterfactual risk cannot be estimated without assumptions about the underlying causal structure of the data. The formula in their Table 1, however, explicitly makes clear that the authors instead compare the factual risk in the full study population with the factual risk in the subpopulation that has remained unexposed. Comparing different populations under the same factual scenario is not the same as comparing the same population under a factual versus counterfactual scenario. Any estimate of this factual estimand will hence be biased when the (implicit) goal is the estimation of the counterfactual estimand.

The importance of formulating precise questions, acknowledging their potential causal nature, and using appropriate statistical methods to answer those questions can be appreciated by the following metaphor. Application of estimation approaches without careful consideration of the estimand of interest is like painting the bull’s-eye around the dart.

Appropriate analytical techniques have been developed for estimation of counterfactual PAFs (3–6); yet, to this day, these are not widely adopted. More problematic is that, in contrast to the PAF, the authors’ targeted factual attributable (excess) mortality measure (“the increase in [death] risk at a given time if one individual is exposed to an event” [p. 162]) does not have a readily apparent counterfactual analogue. It is, for instance, unclear which (counter)factual scenarios the authors desire to compare using this measure, and in which population: the aforementioned scenario in which everyone had remained unexposed with a counterfactual scenario in which everyone had been exposed? Or the factual scenario in the exposed population with the counterfactual scenario in which all exposed patients had remained unexposed? Moreover, each of these contrasts also requires clarification of the role of the timing of potential exposure onset with respect to either the targeted counterfactual scenario or population.

The authors’ work thereby lays bare a pervasive gap in the current medical and methodological literature. To bridge this conceptual gap and to resolve the corresponding ambiguity, researchers need first to decide and agree on where to paint the bull’s-eye (or how to precisely define this causal estimand of interest, in plain English) rather than having their statistical toolbox make that decision.