Peer review in open access scientific journals

The introduction of open access scientific journals has revolutionized scientific publishing. Open access to medical publications has the potential to boost the timely and widespread dissemination of new research findings and, ultimately, to have a more immediate impact on clinical practice than traditional publishing methods. However, several aspects of open access publishing remain to be discussed and clarified by the scientific community; among these is the value and process of open peer review.

The value of peer review to editors and authors is well established in the traditional biomedical publication model. Most authors and editors would agree that a fair and thoughtful manuscript review by experts with considerable knowledge in a given research area is helpful to editors in judging the importance of the research question studied and the appropriateness of the methodology used. Moreover, experienced peer reviewers usually make instructive comments as to whether the authors’ conclusions are supported by their data and whether their findings have been put in the wider context of what is already known about their research question.

However, editors of several influential biomedical journals have expressed concern regarding various aspects of the peer review of submitted manuscripts, including whether generally accepted review processes are adequate to ensure appropriate and fair evaluation, and have called for more quantitative and qualitative studies on the issue.^1-3 Editors sometimes make decisions about submitted manuscripts irrespective of the peer reviewers’ comments; this occurs for a variety of reasons, including the fact that some peer reviews are not particularly helpful.⁴ Nevertheless, traditional peer review has been considered an indispensable part of the editorial evaluation process —until such time as a more effective and fair system is developed and successfully applied in scientific publishing.¹

Although the value of a strong peer review process is felt to extend to the open access publication model,⁵ at least one study has raised the concern that some new open access journals may have a less rigorous manuscript evaluation process than traditional subscription journals.⁶ If this is so, it may reflect an expectation that post-publication review through open, published discourse will be swift and effective. However, while all manner of electronic journals are experimenting with reader input on published material, little is known about the scientific value of post-publication review in the modern era of open access publishing. Several questions arise. Can new capacities for open comment really assume the responsibilities, thoroughness and overall judgement of formal peer review? Are they being set up and structured in this way? It is noteworthy that the multidisciplinary journal Nature closed its recent open peer review experiment, although it should be noted that this “open” movement has differences regarding type of publisher as well access to the data and peer review process compared to that of other publishers such as the Public Library of Science (PLoS) for instance.⁷

In second-generation open access journals such as PLoS ONE there is a more intense emphasis on open post-publication review (after an initial technical review of the paper’s scientific quality) as a means of enabling early publication. This approach makes a greater proportion of submissions available to readers to assess for themselves. Discourse can begin immediately regarding the methodological aspects of a study and the significance and implications of its findings, as compared with traditional journals, where reaction to an article usually takes the form of letters to the editor published at a later date. It should be mentioned that in some first-generation open access journals there has been the opportunity to quickly react to a publication using the “rapid response” electronic system.

In-depth peer review is essential to open access journals no less than to traditional subscription journals both to ensure that the submitted work is within the journal’s scope but also to carefully critique the work before publication. We should not forget that published biomedical information influences clinical practice and thus affects the health of patients. An analogy may be found in the potential dangers for patients if physicians were to make clinical decisions based on results reported in conference abstracts,⁸ given that major differences often exist between data presented in abstracts and subsequent corresponding full text articles published after peer review, revision and editing.⁹ The scientific community should have an unconditional, clear and strong interest in preventing false or over-interpreted scientific information from being disseminated, and this commitment requires a rigorous peer review process.

Open access publications should be at the forefront in experimenting with strategies to foster what might be called an increasingly open science. As the open access movement blossoms, its supporters should continue to critically evaluate the parallel development of openness and transparency in the peer review process. We need to ensure that a commitment to high-quality peer review is maintained. At the same time, it has become clear that some aspects of conventional review processes are ineffective. For example, masking the identity of authors to peer reviewers (blind evaluation) was hypothesized as a way of avoiding conflicts of interest on the part of reviewers, but it has been shown that blinding cannot be always be maintained in either journal manuscripts or conference abstracts.^10,11 Open access journals are in an ideal position to test the merits of open, unblinded, peer review.

Other aspect of openness in peer review also merit exploration. Most manuscripts are rejected on at least their initial submission for publication (especially at the leading biomedical journals). A considerable proportion of these manuscripts are submitted for publication in other journals, usually with lower rejection rates. Of 350 manuscripts rejected by the Annals of Internal Medicine during 1993 and 1994, 69% were subsequently published in other journals with an average time from rejection of the initial submission to its publication in another journal of 18 months.¹² Similar findings are evidenced by law review journals, which permit and encourage multiple submissions and have shorter manuscript handling times than economics, finance, mathematics and psychology journals, which do not allow multiple submissions.¹³ Such publication delays have led to suggestions that simultaneous submission of an article to two or more journals might reduce the time from initial submission to publication.¹³ In addition to reducing delays caused by the requirement for sequential submission, simultaneous submissions may increase healthy competition between journals to reduce their editorial response time.

A natural extension of this “open” submission process would be to make peer reviews of rejected manuscripts available for consideration during subsequent submissions to other journals. Although this practice is recommended by some journals in their instructions to authors, authors rarely adhere to it — for obvious reasons. However, this practice would expedite editorial decisions; for example, it would show whether the authors had addressed previous critiques and thus inform a decision as to whether their work warrants further external peer review. Authors might be more motivated under this system to improve a manuscript after its rejection by paying attention to the suggestions of peer reviewers before submission to another journal, and editors would be able to see where significant improvements to the manuscript have been made.

In conclusion, we know that the peer review process is not without flaws. The birth of the open access publication model and the rise of a more open science presents an ideal opportunity to re-evaluate the transparency of editorial and peer review practices. I believe that when, and if, the scientific community fully adopts policies leading to open access to journal content and the open peer evaluation of manuscripts (e.g., by signing peer reviews and publishing reviews online), the level of transparency and quality in scientific publishing of research data will be further boosted. This will substantially contribute to the advancement of science and to the international community that scientists serve.

Biography

Matthew E. Falagas, MD, MSc, DSc, is Director of the Alfa Institute of Biomedical Sciences (AIBS), Athens, Greece, and Adjunct Associate Professor, Department of Medicine, Tufts University School of Medicine, Boston, Mass.