Abstract
This study investigates potential uncontrolled confounding in meta-analyses of the association of being overweight with all-cause mortality.
Introduction
Is being overweight associated with all-cause mortality, and if so, could the association simply be an artifact of uncontrolled confounding? This question has been controversial, with 2 prominent meta-analyses of nonrandomized studies reporting opposing conclusions. Flegal et al1 reported a protective association of being overweight (but not having obesity) vs having normal weight (hazard ratio [HR], 0.94 [95% CI, 0.91-0.96]), whereas the Global Body Mass Index (BMI) Mortality Collaboration2 (GBMC) reported a detrimental association (HR, 1.11 [95% CI, 1.10-1.11]). Just as individual nonrandomized studies can be biased because of uncontrolled confounding,3 so can meta-analyses.4 In both meta-analyses, many of the included studies did not control for probable confounders (eMethods in the Supplement). We investigated the extent to which potential uncontrolled confounding may have biased these meta-analyses' observed associations.
Methods
To consider how strong uncontrolled confounders in each meta-analyzed study would have to be to negate the observed results, this reanalysis of meta-analyses applied sensitivity analyses for meta-analyses4,5 that are analogous to the E-value3 (analyses conducted on January 7, 2022). First, we calculated an E-value representing the minimum strengths of associations on the risk ratio (RR) scale that uncontrolled confounders would need to jointly have with being overweight and with mortality across all studies in each meta-analysis to shift the meta-analytic estimate or its 95% CI to the null (eMethods in the Supplement).
Second, we estimated the percentage of studies with meaningfully strong HRs (here defined as HR > 1.1 for associations in the detrimental direction and HR < 0.90 for associations in the protective direction), initially without considering potential uncontrolled confounding.5,6 Random-effects meta-analyses accommodate the possibility that studies’ underlying associations differ by estimating the distribution of these study-specific associations; this distribution can then be used to estimate the percentage of studies with meaningfully strong HRs (eMethods in the Supplement).5,6 If this percentage is large (eg, 80%), this may suggest that most studies have meaningfully strong HRs, albeit prior to considering uncontrolled confounding.4,5 We then assessed how strong uncontrolled confounding would have to be to reduce the percentage of meaningfully strong HRs to 15%.5 For code and data for reproducibility, see eMethods in the Supplement.
Results
Before considering uncontrolled confounding, our reanalysis obtained pooled HRs of 0.93 (95% CI, 0.91-0.96; P < .001; heterogeneity τ̂ = 0.13) for Flegal et al1 and 1.10 (95% CI, 1.07-1.12; P < .001; τ̂ = 0.08) for GBMC.2 These analyses comprised, respectively, 140 and 186 estimates from prospective nonrandomized cohorts.
For Flegal et al,1 the E-value suggested that uncontrolled confounders that were associated with being overweight and with lower mortality by an RR of 1.36 each could suffice to shift the point estimate to the null. For GBMC,2 uncontrolled confounders that were associated with being overweight and with mortality by an RR of 1.43 each could suffice to shift the point estimate to the null. To shift each CI so that it included the null, the analogous confounding associations in Flegal et al1 and GBMC2 would be RRs of 1.25 and 1.36, respectively.
Before considering uncontrolled confounding, we estimated the percentage of studies having meaningfully strong protective HRs (ie, <0.9) as 40% (95% CI, 28%-51%) in Flegal et al1 but 0% in GBMC.2 Conversely, we estimated that 50% (95% CI, 34%-63%) of studies in GBMC2 but 9% (95% CI, 4%-15%) of studies in in Flegal et al1 would have meaningfully strong detrimental HRs (ie, >1.1). Again, these percentages refer to the heterogeneous distribution of associations that underlie each meta-analysis. However, small uncontrolled confounding associations (ie, confounding RRs of approximately 1.43 for Flegal et al1 and 1.37 for GBMC2) would suffice for both meta-analyses to bring the percentages of causal effects with HRs below 0.9 (for Flegal et al1) or above 1.1 (for GBMC2) to the same level (15%) in the 2 meta-analyses (Figure).
Figure. Estimated Proportions of Meaningfully Strong Effect Sizes.
The estimated proportions of meaningfully strong effect sizes in each meta-analysis as a function of hypothetical confounding bias in each meta-analyzed study are presented. The x-axis is presented on the log scale, with tick marks on the risk ratio (RR) scale. GBMC indicates Global Body Mass Index (BMI) Mortality Collaboration; HR, hazard ratio; shaded areas, 95% pointwise CIs, which are omitted when they were not statistically estimable (ie, for percentages close to 0).
Discussion
This reanalysis of meta-analyses found that for 2 meta-analyses, uncontrolled confounding associations in each study with RRs of 1.25 to 1.43 could potentially shift the point estimate or CIs to the null or substantially decrease the percentage of meaningfully strong effect sizes. These small degrees of uncontrolled confounding seem plausible in the context of studies’ limited control of confounding by physical, social, behavioral, and psychological factors. These sensitivity analyses suggest that neither meta-analysis provided robust evidence for protective or detrimental potential effects of being overweight on mortality.
Our analysis has several limitations. First, we considered bias of constant severity across studies. Second, thresholds defining associations with meaningfully large HRs (eg, HR > 1.1) are somewhat arbitrary and should be informed by scientific context.5,6 We found that uncontrolled confounding may temper interpretation of meta-analyses. The differing results of Flegal et al1 and GBMC2 may reflect heterogeneous effects and differing inclusion criteria, along with differing uncontrolled confounding. Establishing potentially modest effects of being overweight on mortality would require improved study designs for primary studies and meta-analyses alike.4
eMethods.
eReferences.
References
- 1.Flegal KM, Kit BK, Orpana H, Graubard BI. Association of all-cause mortality with overweight and obesity using standard body mass index categories: a systematic review and meta-analysis. JAMA. 2013;309(1):71-82. doi: 10.1001/jama.2012.113905 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Di Angelantonio E, Bhupathiraju ShN, Wormser D, et al. ; Global BMI Mortality Collaboration . Body-mass index and all-cause mortality: individual-participant-data meta-analysis of 239 prospective studies in four continents. Lancet. 2016;388(10046):776-786. doi: 10.1016/S0140-6736(16)30175-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Haneuse S, VanderWeele TJ, Arterburn D. Using the E-value to assess the potential effect of unmeasured confounding in observational studies. JAMA. 2019;321(6):602-603. doi: 10.1001/jama.2018.21554 [DOI] [PubMed] [Google Scholar]
- 4.Mathur MB, VanderWeele TJ. Methods to address confounding and other biases in meta-analyses: review and recommendations. Annu Rev Public Health. 2021;43:10.1146/annurev-publhealth-051920-114020. doi: 10.1146/annurev-publhealth-051920-114020 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Mathur MB, VanderWeele TJ. Sensitivity analysis for unmeasured confounding in meta-analyses. J Am Stat Assoc. 2020;115(529):163-172. doi: 10.1080/01621459.2018.1529598 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mathur MB, VanderWeele TJ. Robust metrics and sensitivity analyses for meta-analyses of heterogeneous effects. Epidemiology. 2020;31(3):356-358. doi: 10.1097/EDE.0000000000001180 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eMethods.
eReferences.
