Flawed methods and inappropriate conclusions for health policy on overweight and obesity: the Global BMI Mortality Collaboration meta‐analysis

Abstract

Guideline recommendations and health policy decisions rely on evidence from clinical and epidemiological studies. Adequate methodology and appropriate conclusions are essential to support healthcare and health policy decisions. An analysis of body mass index and mortality by the Global BMI Mortality Collaboration (GBMC) concluded that the association of excess body weight with higher mortality was similar worldwide and that overweight and obesity should be combated everywhere. To reach this conclusion, the GBMC used highly selected data, rather than a systematic approach. The GBMC initially chose individual participant data from 239 prospective studies with approximately 10.6 million participants. The GBMC then excluded over 60% of data and over 75% of fatal events by eliminating all cases with any reported disease at baseline or smoking history and all events within the first 5 years of follow‐up. After applying these restrictions, the association of overweight with lower mortality was reversed and the association of obesity with higher mortality was increased. Given the major flaws in the selection process, in the adequacy of the data, in the data analysis, and in the interpretation, the GBMC conclusions should be viewed sceptically as a guide to action, either for clinical decisions or for public health in general. The flawed conclusion that overweight is uniformly associated with substantially increased risk of death and thus should be combated in any circumstances may lead not only to unjustified treatment efforts and potential harm in a wide range of clinical conditions but also to a tremendous waste of resources.

In 2016, the Global BMI Mortality Collaboration (GBMC) published a meta‐analysis1 of body mass index (BMI) and all‐cause mortality that has already been cited 378 times as of September 2018. This highly visible publication was authored by a writing committee of 61 distinguished epidemiologists and included over 500 collaborators.

The data selection

The publication was a meta‐analysis but not a systematic review. Examinations of the GBMC article revealed numerous inconsistencies and flaws in the way the data were selected.2, 3, 4 As we discuss in detail elsewhere,3, 4 the data selection appeared to consist only of articles already known to the senior GBMC author and whose results were also largely already known. Some data meeting the inclusion criteria were excluded; in turn, some data meeting the exclusion criteria were included. Some of these inappropriate inclusions and exclusions could have changed the essential findings of the meta‐analysis. Appropriate data sets, some even containing millions of additional participants, were not sought out or included.2, 3, 4

The data restriction

The analytic methods used in the GBMC article were flawed. The original sample consisted of 239 studies with a combined sample size of 10 625 411. However, the authors deleted over 60% of the data that they considered and about 75% of the deaths to arrive at their final results. The final sample used for the primary pre‐specified analysis after applying the planned restrictions consisted of 189 studies with a sample size of 3 951 455. These analyses are described repeatedly as ‘strict’ or ‘stricter’ primary analyses, using these words six times in the article and another six times in the supplement. This word choice implies that carefully selected criteria were rigorously applied. However, the GBMC analyses were far from ‘strict’ in this regard. The GBMC paper in fact, had very limited ability to standardize covariate or even exposure definitions and could not standardize the data used for the listed exclusion criteria, as discussed below.

The GBMC article claims that ‘The primary analysis was restricted to never‐smokers without certain known chronic diseases at baseline (e.g., cardiovascular disease, cancer, or respiratory diseases), and omitted the first 5 years of follow‐up’. Because of the lack of complete data on smoking or pre‐existing disease, however, the primary analysis could not even perform the claimed exclusions adequately. Only 28 of the 239 studies even had complete data for the three chronic diseases mentioned previously (see eTable 2 in GBMC1). Twenty studies had no information on any of the pre‐existing diseases. Only 15 of the 239 studies provided data on respiratory disease and only 68 had data on cancer. Only 19 studies had complete data on pre‐existing heart disease, stroke, and cancer. Only 56 studies had data on both cardiovascular disease and cancer.

A major exclusion criterion of the primary analysis was smoking status. Most studies in the GBMC publication could not identify never‐smokers adequately (see eTable 6 in GBMC1). Only 98 of the 239 studies were able to distinguish between never‐smokers and current or former smokers. Thus, the GBMC analyses were not able to appropriately adjust for smoking or pre‐existing disease because the researchers were unable to harmonize either smoking or pre‐existing disease data across studies.

The GBMC article asserts: ‘To help achieve more valid estimates, prospective studies of BMI and mortality should, when possible, exclude: smokers, participants who already have some chronic disease at recruitment that could affect BMI, and those dying within 5 years of recruitment’.5, 6, 7, 8 To justify the claim that these restrictions help to achieve more valid estimates, the authors cite a letter to the editor7 and three articles.5, 6, 8 None of these citations have any data to support the statement that the restrictions used in the GBMC restrictions will decrease bias. Two5, 7 are simply opinion statements written by several authors of the GBMC article, one is a simulation study6 with no data and which actually does not reach this conclusion, and one8 does not address bias at all. Thus, the GBMC citations do not support the claim that these restrictions produce more valid estimates. These ‘restrictions’ seem to yield estimates that the GBMC senior investigators apparently believe to be correct. Expert opinion is a common category in schemata of levels of evidence used in guidelines and recommendations. However, it is usually considered the lowest level of evidence. It is important to clearly identify conclusions based on expert opinion and not to mask them with inconsistent, selective, and over‐interpreted data.

Previous analyses by GBMC authors of the data sets used in the GBMC article did not use these exact restrictions, in particular those involving the deletion of initial years of follow‐up. The GBMC analyses deleted the first 5 years of follow‐up, but other analyses of subsets of the same data by the same authors used either no deletions for follow‐up,9, 10, 11, 12, 13 or less than 5 years.14, 15, 16, 17, 18 Only Chen et al.19 and Whitlock et al.20 deleted 5 years. No justification is given why 5 years should be excluded, and no sensitivity analyses are provided on what the effects would be of using different lengths of exclusion or no exclusion of any follow‐up.

There is no evidence that the GBMC methods are successful in controlling bias. First, even their attempts to control for smoking and pre‐existing illness were inadequate because most of their studies either lacked or did not report the necessary data. Second, although the restrictions changed the results, there is no evidence that this is because they were effective at controlling bias.

Unaddressed bias

In contrast to the debatable and inconsistent restriction procedures as outlined previously, the GBMC analyses ignored a major obvious source of potential bias, namely, the use of self‐reported rather than measured weight and height. Unlike the other sources of potential bias that they tried to control for, there is ample documentation of systematic errors in self‐reported weight and height, leading to systematic bias, errors in BMI, and misclassification into the correct BMI categories relative to measured weight and height.17, 21, 22, 23, 24, 25, 26 In the primary GBMC analysis of North American data, 94% of deaths came from 20 studies with self‐reported weight and height. The GBMC found that for North America, the hazard ratio for overweight was 1.00 using measured weight and height and 1.14 using self‐reported weight and height. BMI calculated from self‐reported weight and height is systematically biased relative to BMI calculated from objectively measured data,22 despite a high correlation between the two. The errors in self‐reported weight and height can lead to substantial misclassification into BMI categories and attendant bias with overestimation of hazard ratios.21, 23, 24, 26

The errors in BMI classification from self‐reported weight and height are known to vary by multiple potential confounding factors such as age, sex, race, socio‐economic status, health conditions, region, and language.27 Unmeasured residual confounding is also likely to vary unpredictably across studies and thus can bias not only individual studies but also comparisons between subgroups and regional comparisons. Because of the high potential for misclassification and unmeasured confounding associated with self‐report, an easily accomplished first step, if truly more ‘strict’ control for bias had been intended, would require excluding studies with self‐reported height and weight and limiting analyses to studies with measured data. Unfortunately, the GBMC article excluded large numbers of available data sets with measured height and weight that either were not in the initial group of 239 studies that were considered or were not included in the group of 189 studies used in the final analyses. Their final analyses included far more participants with self‐reported weight and height than with measured weight and height. The GBMC purports to draw inferences about BMI categories, but such inferences can be quite inaccurate because the rate of misclassification into the wrong measured BMI categories is high when self‐reported weight and height data are used.25 The effects can be strong enough to change the direction of hazard ratios for some categories and particularly the debated overweight category.23

The discussion in the context of other published studies

The GBMC article claims, incorrectly, that ‘our findings are also broadly consistent with the stricter analyses done in a 2015 study of 12 million Korean adults’ by Yi et al.28 This is a huge study with larger sample size than all 239 GBMC datasets combined but which was nevertheless not even considered by GBMC. The estimates from the GBMC paper are actually not consistent, either broadly or narrowly, with the Korean study. In fact, Yi et al. found almost no effect of applying the GBMC restrictions, in sharp contrast to the GBMC claims that the restrictions drastically change (‘correct’) the results. Yi et al. in particular still found hazard ratios below 1 for overweight subjects, suggesting that in fact, the GBMC findings are misleading and the applied restrictions introduced bias to the GBMC data.29 The Yi paper is instead consistent with the findings of Flegal et al.30 In particular, the data from Yi et al. showed that overweight was not associated with an increased risk of death (hazard ratio of 0.85) and grade 1 obesity (BMI 30–35) was associated with only a very small increased risk of death (hazard ratio of 1.06).

Unjustified generalization of findings

The GBMC report claims that its findings support strategies to combat the entire spectrum of excessive adiposity across all populations, but this is not clearly justified. The primary pre‐specified analysis excluded (where identifiable) ever‐smokers, excluded those with some prevalent disease (again, where identifiable), and required a survival of at least 5 years. The conclusion from these data thus would at best only be applicable to healthy, never‐smoking individuals with a life expectancy of more than 5 years. Furthermore, the data are inadequate to even support those exclusions, and the exclusions lack a clear rationale. The findings cannot, and must not, be applied to patients with any chronic disease setting where long survival is uncertain.

In this context, the GBMC authors incorrectly quote previous work on protective effects in supposedly healthy individuals.31 Contrary to what they claim, the cited study did not address apparently healthy individuals but clearly referred to patients with established cardiovascular disease. There is substantial and constantly growing evidence from multiple studies in wide ranges of disease states and in numerous disease settings and using many ways of analyses, that shows that once a chronic disease is established, overweight (BMI 25–30) or mild obesity (BMI 30–35) is associated with incremental longevity as compared with those patients with a normal BMI (18.5 to <25).32 The extra weight (if ‘extra’ is even a proper term here) may protect these patients against ravages of diseases such as heart failure, advanced chronic kidney disease,33 terminal cancers, as well as sarcopenia of old age.34 Arguments that this is just an artefact of selection bias are undermined by the observation that the same phenomenon is seen even for chronic diseases that are not obesity‐related.35

The uncritical translation of findings in healthy non‐smoking individuals to patients with established chronic diseases exemplifies a common misconception about the significance of weight reduction in such patients. Promoting weight reduction in patients with established chronic disease goes against clinical evidence and guidelines36 and may be harmful for patients.

Applying emerging data in clinical decision‐making—evolving from paradox to paradigm

The finding that overweight and mild obesity are associated with better outcome in a range of diseases was initially termed ‘obesity paradox’ because it appeared to be an unexpected and counterintuitive finding. This terminology has led to misunderstandings among researchers and the public alike. There is no precise definition to the term, and numerous and sometimes loosely related observations have been summarized under this seemingly paradoxical finding.35 The core observation, however, of an inverse association of body weight with outcome in patients with prevalent diseases has been confirmed in multiple reports and in numerous patient cohorts, in different ethnic groups,37 over a wide spectrum of cardiovascular diseases and disease severity and using various methods. With an increasing bulk of data confirming a survival benefit related to overweight and mild obesity in such diseases, and based on pathophysiologic concepts to explain these findings, there is sufficient scientific evidence to abandon the term ‘paradox’. Importantly, a finding considered ‘paradoxical’ will not be applied in clinical practice as it implies an unexplained if not unscientific statement. Accordingly, in recognition of the robust evidence that has been accumulated and the plausible pathophysiologic concepts, it has been suggested that an obesity paradigm should be defined rather than a paradox.38 This advance in terminology will allow the application of the evidence in clinical practice and promote a highly needed differentiated body‐weight management that appreciates individual health conditions.

Potential impact of misleading conclusions on health care policy

In their sweeping conclusions, the GBMC authors claim that their results apply globally, but in fact, of the five regions they studied, the results are quite different for two of the five regions compared with the other three. A core issue is whether the overweight category has increased mortality risk. The 2013 US obesity guidelines,39 sponsored jointly by the American Heart Association, the American College of Cardiology, and The Obesity Society, assert that overweight is not associated with excess mortality, citing the strength of the evidence as ‘moderate’. The European Society of Cardiology Guidelines on diagnosis and treatment of acute and chronic heart failure state that in patients with heart failure with moderate degrees of obesity (BMI < 35), weight loss cannot be recommended.36 In contrast to the majority of available evidence, the authors of the GBMC report concluded with a recommendation to combat any degree of excessive body weight in any condition where it is found. This erroneous conclusion has major implications because a huge number of people are in this category. In most countries, people in the overweight category outnumber those in the obesity category. Analyses of burden of disease that were based on the estimates of risk from the GBMC analysis were prominently published in the NEJM.40 They calculated that 4.0 million deaths annually worldwide are due to high BMI, and nearly 40% of them (1.5 million deaths) are in people who are not obese but only overweight.

It makes a huge difference if the annual number of deaths due to overweight is 1.5 million, as the GBMC estimates would suggest, or close to 0, as the totality of the evidence suggests. In the first scenario, a major investment of public health effort should be devoted to treatment and prevention in the overweight category. In the second scenario, such an investment of public health effort would be catastrophic: no benefit would be gained in the overweight category, and resources would shift away from obesity per se and underweight, where major increased risks of death are clearly documented. The lack of documentation of whether most currently available preventive and therapeutic interventions are sufficiently effective even in obesity,41 let alone overweight, adds further to the potential for major waste of resources.

Given the major flaws in the selection process, in the adequacy of the data, in the data analysis, and in the interpretation, the GBMC conclusions cannot be trusted as a guide to action, either for clinical decisions or for public health in general.

Conflict of interest

The authors declare that no conflict of interest relevant to this article exists. The authors certify that they comply with the ethical guidelines for publishing in the Journal of Cachexia, Sarcopenia and Muscle: update 2017.42

Flegal, K. M. , Ioannidis, J. P. A. , and Doehner, W. (2019) Flawed methods and inappropriate conclusions for health policy on overweight and obesity: the Global BMI Mortality Collaboration meta‐analysis. Journal of Cachexia, Sarcopenia and Muscle, 10: 9–13. 10.1002/jcsm.12378.