Quality Management in Scientific Publishing – the Importance to Critically Scrutinize Scientific Work

On September 10, 2013, an article with the title ‘Unbalanced Baseline in School-Based Interventions to Prevent Obesity: Adjustment Can Lead to Bias?’ by the authors R. Sichieri and D. B. Cunha was submitted to our journal OBESITY FACTS. The reviewers of the original submission recommended that major revision was required before the publication could be accepted, specifically a stronger defense of the criticism of baseline adjustment and pooled analysis confirming or rejecting the authors' hypothesis were recommended. On November 8, 2013, the authors submitted a revised version which, after reconsideration by the original reviewers, was then accepted for publication. The paper was published online on June 28, 2014 and is part of issue 4, 2014 of OBESITY FACTS[1].

At the end of September 2014, a Letter to the Editor ‘The Assertion that Controlling for Baseline (Pre-Randomization) Covariates in Randomized Controlled Trials Leads to Bias is False.’ by P. Li et al. commenting on this article was submitted to OBESITY FACTS and accepted for publication in the present issue [2]. The authors expressed fundamental and severe criticism with regard to the above mentioned article that culminated in the conclusion that the article should be retracted. The authors of the criticized article were asked to respond to the criticism and to meet the request that their article should be retracted. The answer of Sichieri and Cunha to Li et al. and their reasons for refusing to retract their article are also published in this issue [3].

As the criticism expressed in the Letter to the Editor is so severe, an independent statistician was asked to review the original article and the subsequent correspondence. The statistician agreed with the criticism raised by Li et al. [2] and concluded that the critical comments were not adequately addressed in the Authors’ Response [3]. Specifically, Sichieri and Cunha [1] incorrectly assume that the same issues regarding adjustment for baseline values in observational studies apply to cluster randomized trials. They state in their Author's Response that ‘there are no references related to experimental designs because … by design we expect lack of bias’. The counter-argument is that there are no references because there is no possibility of bias in the randomized setting. The main point of the Author's defense is that they had performed a study in which they had obtained different results depending on whether or not baseline was adjusted for in their mixed model analysis. However, after reviewing these results, there are concerns about the models fitted. (It appears that time was modelled continuously, which is not appropriate with only three time points [4]. Further, in their baseline adjusted analysis only post-baseline values should be included in the outcome measure, and it is not clear whether this is the case or not.) Even assuming that the models they fitted are appropriate, the magnitude of their point estimates for their coefficient of interest do not differ greatly between the two models. Furthermore, as pointed out in the Letter to the Editor [2], their meta-analysis does not provide any evidence for or against the presence of bias. By simply stating that there were a high number of studies with imbalance, Sichieri and Cunha [3] do not address to the question of whether or not adjusting for this imbalance results in bias.

By publishing the Letter to the Editor and the Authors’ Response together with this Editorial in the current issue of OBESITY FACTS, I would like to open the discussion that arose between the authors of the original article, the authors of the Letter to the Editor and the Editor-in-Chief to our readership and recommend reading and forming your own opinion.

Stimulated by this debate, I would like to point out that it is each author's responsibility to make sure that statistical procedures are correctly used and valid for the study submitted. Even though it would be desirable if an additional assessment of statistical methods could be established within the review process, it cannot be expected that all of our reviewers, in addition to their expertise in various aspects of obesity research, are designated experts for advanced statistical procedures.

Finally I would like to emphasize the importance of critically scrutinizing scientific papers even after their publication. Therefore, comments on articles are highly welcomed at all times to present and discuss different viewpoints concerning a controversial topic.