Imperfect but important: a fellow’s perspective on journal peer review

“Anyone who reads journals widely and critically is forced to realize that there are scarcely any bars to eventual publication. There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.”

-Drummond Rennie, 1986 [1]

Written over 30 years ago by then-senior contributing editor of JAMA, some argue this statement still describes the world of medical publishing. If anything can be published, what is the point of peer review? And why would any reputable journal use it?

Dr. Rennie poses this question to himself in that very commentary and ultimately decides that the peer review process should determine where, not which, manuscripts will be published and to improve the quality of those works [1]. JMT holds to this goal, but admits that the process can be opaque and potentially frustrating for authors, especially those new to publishing. The system can, however, be made clearer if one has the opportunity to peek behind the editorial curtain. To that end, here I hope to share the insights of a JMT FIT editor so that readers may understand better the journal’s process of and justification for using peer review.

Manuscript review at JMT involves editorial board review first and then assessment by two to four external “peer” reviewers blinded to the identity of the authors. Peer reviewers are asked to read manuscripts methodically and provide timely, organized, and constructive assessments to help writers clarify their message for eventual readers [2–5]. The quality of this feedback can be variable as almost no one is formally trained in peer review. JMT’s website does include a freely available webinar [6], and the publisher provides online resources for those interested in providing optimal reviews [7], but admittedly, most reviewers do not use these references. They learn instead through practice and model their comments on what they have seen in the decision letters they have received as authors. JMT, like other journals, hopes its reviewers follow the “golden rule” and provide the kind of feedback they would themselves like to receive as authors. Naturally, some reviewers take this message more to heart than others, although, by and large, a majority of reviewers do follow the principle. (Of note, as someone who has used and benefited from the aforementioned resources, I would recommend them to both new and seasoned reviewers.)

When evaluating submissions, reviewers should not limit their comments solely to the scientific methodology and quality of writing; they should also consider the ethical implications of how the work was performed and will be disseminated. Specifically, reviewers may comment if a manuscript message seems biased by undisclosed conflicts, manipulates data in unfair ways, or plagiarizes the work of others. Although JMT’s editors use tools such as iThenticate to examine the originality of submitted manuscripts, reviewers share the responsibility for identifying problems [8]. The discovery of an ethical concern during review is not necessarily the kiss of death for a manuscript. Editors understand that in the modern, fast-paced era of electronic dissemination of knowledge, Free Open Access Medical Education (FOAMed), and the common tendency to use “copy-and-paste” when writing, errors when writing may reflect honest blunders, not intentional deceit. If a mistake is born of haste and not malintent, authors are allowed to the opportunity to explain and correct the manuscript during revision.

Of note, not all manuscripts submitted to JMT undergo formal peer review. Abstracts from national meetings, position statements, invited commentaries, and letters to the editor do not require external review; they are evaluated by one of the editors for clarity and formatting alone [9]. Sometimes manuscripts submitted to JMT simply do not fit the journal’s scope and mission. In these cases, rather than sending the manuscript out for peer review and having the author wait weeks for an inevitable rejection, the editor-in-chief or section editor will issue an immediate decision. In this way authors may resubmit to a more suitable journal without delay.

The foregoing justification of peer review notwithstanding, it is important to acknowledge that the process is far from perfect. Authors grumble, justifiably, that after months to years of toiling away on a given project, receiving fastidious feedback from a disembodied, anonymous critic who invariably requires them to do more work can feel antagonistic. When comments are disorganized or overly critical, the review may feel downright demoralizing. Critics of the traditional peer review process have echoed authors’ complaints, describing it as cumbersome, slow, and of questionable efficacy. Even editors of some prominent international journals have suggested it be abandoned and replaced with alternatives such as non-blinded review, post-peer review, and publication without review. In the words of one clever expert, “ If peer review were a drug, it would never get on the market” [10].

These suggestions, while not unreasonable, are often grounded in the assumption that publication represents a fait accompli, and that peer review is a gold stamp of approval. JMT, however, holds that “just because it’s published, doesn’t mean it is perfect” [11]. Peer review and publication are but two steps in the ongoing process of introducing new knowledge to the bedside. Journal readers’ criticism and dialogue through journal clubs, social media, and letters to the editor, are equally important in vetting published works. JMT aims to do the best it can with what is an inherently imperfect process, but all should remember the process of review does not and should not stop once something is published.

Certainly, the review process is flawed. It can be frustrating, it can be slow, and it can be wrong. But there is no philosopher king in the world of peer review; there exists no infallible, omniscient observer to provide timely and completely objective criticism. Peer review must therefore be a collaborative effort on the part of readers, reviewers, and authors, most of whom are one and the same. It is shepherded by an editorial board which should, and here does, encourage feedback and transparency. And it is when all parties engage positively in this process that peer review at JMT may come closest to achieving its stated goal: to enhance authors’ works and ensure that the journal thrives as a preserve of ethically-produced pieces of quality and substance that will impact clinical medicine for years to come.

Compliance with Ethical Standards

Conflicts of Interest

None