Effects of being overweight on mortality are unclear given multiple methodological problems

To the Editor:

We thank Katherine Flegal for her response¹ to our recent article on potential uncontrolled confounding in estimated effects of being overweight (but not obese) on all-cause mortality.² Our paper did not aim to “explain the differences between … [estimates from] Flegal et al.³ and … the Global BMI Mortality Collaboration (GBMC)⁴”, nor to conclude that such differences reflect uncontrolled confounding. Rather, we concluded merely that neither pooled estimate was especially robust to uncontrolled confounding. Indeed, we explicitly left open the question of why the estimates differed: “The differing results of Flegal et al.⁵ and GBMC ⁴ may reflect heterogeneous effects and differing inclusion criteria, along with differing uncontrolled confounding.” Flegal’s response¹ does a fine job of suggesting possible explanations.

Like Flegal,¹ we found that GBMC’s⁴ criteria for including studies were unclear and could not be reconstructed from information in the paper or its supplement. Moreover, it is troubling that Flegal’s previous requests for a transparent statement of the inclusion criteria remain unaddressed.³ We, too, are concerned by analyses that exclude participants who died early⁴ because conditioning on a post-treatment variable can introduce bias. Unfortunately, this practice is not uncommon in the literature on body mass index (BMI) and mortality, including in a number of studies analyzed in Flegal’s⁵ meta-analysis.

We showed that considering even one major potential source of bias – namely uncontrolled confounding – suggests that neither meta-analysis provides strong evidence given the small estimated effect sizes and the often poorly controlled designs of primary studies.² It is striking that, in this literature on a prominent public health topic and that comprises well over 100 studies, most studies⁵ or all studies⁴ did not adjust for probable confounders such as socioeconomic status, physical activity, dietary quality, and baseline BMI. As Flegal has correctly noted,⁵ many variables that might strongly confound the relationship between being overweight and mortality, such as comorbidities and lifestyle factors, probably also constitute mediators. In both meta-analyses,^5,4 many of the studies (perhaps all) measured such covariates at the same time as the exposure, which can lead to bias in either direction.

We would suggest several methodologic priorities for this literature. Primary studies should control for a rich set of plausible confounders measured prior to BMI, ideally including past BMI. The practice of excluding participants who died early should be replaced with principled adjustment for pre-exposure indicators of underlying disease and its correlates. In meta-analyses, inclusion criteria should be preregistered and transparently reported according to existing guidelines. These inclusion criteria should be designed to minimize within-study biases, including uncontrolled confounding.⁶ Meta-analysts should report detailed risk-of-bias assessments and sensitivity analyses⁶ and, when the primary evidence is insufficiently rigorous, should convey overall conclusions with appropriate circumspection.

Overall, we consider the question of whether being overweight (but not obese) affects all-cause mortality to be largely unresolved, though we suspect that effects are typically small and heterogeneous across populations. It is rather puzzling that public health messaging has almost exclusively discussed detrimental effects when in fact, there is currently not particularly robust evidence for effects in either direction.