Who watches the watchmen and the problem of recursive flea bites

When the Roman poet Juvenal asked ‘Quis custodiet ipsos custodes?’ (‘Who watches the watchmen?’) he was questioning the integrity of guards posted to ensure his wife's fidelity. More recently the phrase has been generalised to challenge authority, the utility of regulators, and schemes of quality assurance.

In this context Avidan and colleagues¹ challenge the traditional IMRaD design of scientific papers (Introduction, Methods, Results, and Discussion) and focus on whether we can improve the structure and function of the discussion section in scientific publications. After succinctly reviewing the shortcomings of a process in which authors discuss their own findings, point out their own shortcomings, and selectively review the existing literature, they suggest adding a second discussion section penned by an independent ‘Discussant’. With full access to the protocol and study data but no sight of the authors' own discussion, the Discussant would independently evaluate the meaning of the research findings, and possibly conduct a literature review, repeat or extend the analysis of the raw data, or both. In a worked example, a further independent individual compares the two discussion sections and evaluates the implications of their consistency or otherwise.

And so, on to flea bites. The 19th century British mathematician Augustus De Morgan digressed from constructing the rules of logic to pen the poem Siphonaptera, which opens ‘Big fleas have little fleas upon their backs to bite 'em, and little fleas have lesser fleas, and so, ad infinitum.’ Setting aside the question of plagiarism (De Morgan stole the idea from the 18th century satirist Jonathan Swift), we need to face the reality of resources.

Editors already struggle to find reviewers for journal manuscripts, and sourcing reviewers for grants is an order of magnitude harder. Adding independent reviewers and Discussants to the roll call for each paper published implies flogging the same pool of reviewers harder or (implausibly) recruiting fresh blood. This seems unlikely. The principal motivations for review activity are probably personal development, networking, and personal satisfaction rather than academic advancement—as universities celebrate grant income and high impact publications and non-academic employers are generally indifferent.

Avidan and colleagues¹ suggest that the work of a Discussant might be recognised with authorship—not a bad idea, but one likely to precipitate an arms race fuelled by this recognised academic currency. If Discussants are to be authors, then why not the hard-working reviewers? Who would slave away writing (anonymously) carefully crafted and hopefully supportive reviews when for a similar amount of work as a Discussant (or perhaps a bit more), they could become an author? Worse, as a candidate Discussant, I would be tempted to accept invitations to become Discussant on important manuscripts from well-known authors and quite uninterested in the same role against less exalted academic material.

If inadequate discussion is indeed a problem (and it probably is), then what else might we try to improve it? As is often the case, a good start might be the consistent enforcement of existing rules. The British Journal of Anaesthesia already provides in its Instructions to Authors www.journals.elsevier.com/british-journal-of-anaesthesia, a comprehensive checklist for elements to be considered in discussion. This could be expanded to become a reporting template along the lines of the widely used Structured Abstracts presented in the IMRaD format. In parallel, reviewers could be tasked with using the same set of discussion headings, thereby ensuring a consistent approach. We already have a Consolidated Standards of Reporting Trials (CONSORT) statement, checklist, and flowchart for reporting clinical trials www.consort-statement.org, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) www.prisma-statement.org, and equivalent guidance in other domains. Volunteer Discussants are never going to have the time or necessarily the expertise to reanalyse original data; where appropriate, this could be undertaken by a paid professional statistical reviewer.

Moving up a level, we should consider the use of editorials. When a peer reviewer reports to the Editor on a submitted manuscript, there is the option to suggest commissioning an accompanying editorial. Reviewers can identify manuscripts whose contents are especially impactful, surprising, or controversial and the editorialist—if one can be found—draws out these elements, sets them in a wider context, and suggests implications for practice, future research, or the health system, as appropriate. Arguably, much of the third-party perspective proposed for Avidan and colleagues'¹ Discussant is already available from well commissioned editorials. Good science needs to be published in a timely manner and editorials are therefore often written at pace. A more reflective analysis with a less pressurised timeline is available from (hopefully) expert and unbiased editorial reviews, which can be either commissioned or unsolicited.

Discussion sections do indeed represent the weakest element of the journal paper and there is certainly scope for improvement. Perhaps it is time for some experiments?

Declaration of interest

The author declares that they have no conflict of interest. JRS reviewed the manuscript by Avidan and colleagues and wrote this editorial at the request of the Editor-in-Chief.