Review of publication bias in studies on publication bias

Editor—Dubben and Beck-Bornholdt conclude that there is no evidence of publication bias in reports on publication bias.¹ Apart from the fact that funnel plots should be used only as a “tool” and not a “rule” in the evaluation of publication bias,² I question whether such “meta-research” really helps to improve patient care or facilitates the applicability of research results.

Systematic reviews help to improve patient care since pooling of appropriate data sometimes enables us to see the results without the noise of the random play of chance. All sources of bias are a potential threat to the credibility of meta-analyses. Despite efforts to ensure that the set of trials used in meta-analyses is a non-biased sample of all existing studies, a recent analysis on studies in the Cochrane database finds that publication bias may be present to some degree in about 50% of meta-analyses and strongly indicated in about 20%.³

What are the implications of these findings? When such facts are known, doubts arise on the applicability of such “skewed data.” It is even harder to imagine the implications of the meta-research presented in by Dubben and Beck-Bornholdt.

Where do we go from here? Do we need another investigation that focuses on the biases associated with the publication of papers on the “publication bias in studies on publication bias”? What do the results mean other than that publication bias is no fiction: it exists in the real world.

For the clinician it may be reassuring to know that, in most cases, these biases did not affect the conclusions.³ Therefore, systematic reviews continue to represent a valuable tool to digest huge amounts of research or to find hidden “pearls of evidence” given the time constraints in daily business. This enables clinicians to focus on the “complicated, value laden, rewarding activity that is clinical medicine.”⁴