Adjusting the use of preprints to accommodate the ‘quality’ factor in response to COVID-19

Abstract

Preprints are typically crude precursors of peer-reviewed papers that are placed almost immediately, save for some superficial screening, on an open-access repository to allow the information to reach readers quickly, circumventing the long-drawn process typically associated with processing in peer-reviewed journals. For early-career researchers who might be enthusiastic about obtaining some recognition for their efforts, or wanting open and public input about their work, preprints are certainly a useful publication choice. However, if health-related data and information have not been carefully scrutinised, they may pose a risk and may even serve as a source of public health misinformation. Surging growth and competition among preprint servers, coupled with a massive volume of COVID-19-related preprints, mainly on bioRxiv and medRxiv, as well as select indexing now being tested on PubMed, suggests that preprints are being increasingly used in the biomedical sciences. Stronger and more robust ethical policies are needed to screen preprints before they are released to the public, and even if this implies a slight delay in publication, it may increase academics' trust in this form of scientific information and communication. Clear and stringent ethical policies need to be urgently introduced by ethics groups such as COPE and the ICMJE, whose many member journals allow preprints to be posted before traditional peer review. Stringent ethical guidelines that treat misconduct equally in preprints and peer-reviewed papers will boost the integrity of academic publishing.

Introduction: Battle for dominance of preprint market

Preprints are ‘a crude document representing information that has yet to be critically scrutinized by peers’ (p. 1026).¹ The discussion about preprints continues to swirl around the issue of ‘quality’ control and the implications for citations of these non-peer-reviewed documents. A preprint is, by nature, a paper in a raw state of development and analysis. Following feedback from peers and the public before, or concurrent with, submission to a peer-reviewed journal, it can undergo alterations, general improvements, and enhancements, and can be gradually revised, evolving into new versions as it seeks a home in a preferably peer-reviewed journal. Alternatively, it can remain idle, with a suspended fate, as a preprint, with suspended – and untested – ‘quality’. It is unclear how many preprints do not eventually make it into the peer-reviewed literature, or are not cited. These are subjects that are worthy of future analysis. However, some preliminary evidence already suggests that a solid percentage of published papers that were preceded by preprints garner higher altmetric scores than papers that were published without taking the preprint route.² Fortified altmetrics, citations, and social media attention have been found for bioRxiv³ and arXiv^4,5 preprints. Preprints are frequently touted as tools to promote the work of early-career researchers, many of whom see this form of publication as a rapid and easy way to showcase their developing work,⁶ or to enhance the speed and efficiency of scientific exchange.^7,8 Apart from greater exposure and awareness of work via preprints, peer-reviewed papers that have passed through the preprint route ultimately appear to garner more citations.⁹

This paper discusses the issue regarding the citation of preprints from a slightly adjusted philosophical perspective to that put forward by the author a few years ago.¹ In that paper, and at that time, preprint platforms or servers such as preprint.org or bioRxiv had just begun to take root, and the entire premise of the use of preprints within the publishing pipeline was still new and in a state of heated discussion and evolution.¹⁰ The notion of preprint ‘wars’ was proposed, suggesting that proactive competition by competing parties had begun for an expanding preprint market.¹¹ Since preprints represent the first step in the gateway towards publication in a peer-reviewed journal, many of which are either for-profit subscription journals or open-access journals that charge an article-processing fee, the notion of preprint ‘wars’ – virtual competitive battles for an intellectual and financial reward – is neither unreal, nor far-fetched.

Not that long ago, the ‘acceptance’ of preprints among many mainstream journals and publishers was still not absolute; in other words, there was still a sector of the publishing community that actively rejected preprints, or that did not want to accept preprints for peer review, considering them to be ‘prior’ publications.¹² Fast forward to 2019, about six months before the first cases of COVID-19 emerged in China, medRxiv was launched by Cold Spring Harbor Laboratory (CSHL), Yale University, and BMJ as a preprint server for the medical sciences. The philosophy about the risks shared in 2017 has, in fact, not changed much, as certified by medRxiv itself, which posts a notice on the top page of all preprints stating, ‘Caution: Preprints are preliminary reports of work that have not been certified by peer review. They should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information’. In other words, there are risks in using, citing, or relying on preprints as ‘documents of fact’. Some of those risks include the invalidation of claims or the lack of validation of claims, or the possibility of using or citing unscrutinised information, resembling ‘predatory’ publishing, whereby anything can be published or cited without scrutiny; the ease with which conflicts of interest can be hidden (or not); and the extended gaming of metrics.^13,14 Despite the advances made in the preprint ‘market’ by medRxiv, some members of the medical community continue to actively resist the acceptance of preprints related to clinical research for subsequent peer review.^15,16 Their voices and concerns cannot, and should not, be ignored. Although preprints are gaining favour, a sector of academia remains resistant to this form of publication, the main issue being credibility, or the lack thereof.¹⁷

Preprints in the COVID-19 era: Ethical challenges

Fast forward once again to 2021. With over 129 million cases and 2.81 million deaths worldwide due to the COVID-19 pandemic,¹⁸ CSHL's bioRxiv and medRxiv have now become the ‘leading’ (in terms of volume) preprint servers in biology and medicine, with 14,682 preprints related to COVID-19 or SARS-CoV-2, the virus causing this disease, published to date (11,379 in medRxiv, 3,303 in bioRxiv:).¹⁹ In some ways, almost ironically, COVID-19 ‘aided’ the projection of these two preprint servers to ‘success’ (i.e. increased use) and preprint market dominance. Other preprint servers or service providers, such as the suite of 26 preprint servers by the Center for Open Science (COS) hosting 2.297 million preprints,²⁰ do not directly ‘compete’ thematically with bioRxiv's and medRxiv's preprint biology and medical science market prominence, which took just under eight years to be established (bioRxiv launched in 2013), and their prominence was cemented in the COVID-19 era. An assessment of preprints in the first four months of 2020 indicated that 15% of abstracts in COVID-19-related papers underwent minor alterations by the time they had been published in peer-reviewed journals.²¹ A separate assessment of preprints in the first four months of 2020 found that the average time for a preprint to become a published paper was 63 days.²² Another study found that peer-reviewed COVID-19 papers took an average of 83.8 days between submission and publication, relative to 199.7 days for non-COVID-19 papers and 201.7 days for pre-COVID-19 papers.²³

There is a real and tangible risk of misinformation on human health, including exaggerated claims and hype,²⁴ even more so now during the COVID-19 pandemic.²⁵ That risk can emerge from preprints,²⁶ peer-reviewed literature,²⁷ or predatory publishing venues,²⁸ both open-access or subscription, even more so given the deluge of COVID-19-related literature being published.²⁹ One new and extremely serious risk in preprints is their silent (complete or partial) withdrawal or retraction from the published public record without any suitable explanation or transparent reason.³⁰ If such papers were to be peer-reviewed literature, such silent retractions/withdrawals would violate retraction policies by the Committee on Publication Ethics (COPE). However, preprint servers are not currently – as far as the author is aware – COPE members, and since COPE also apparently does not clearly adjudicate the ethics or provide ethical guidelines pertaining to the retraction of preprints, despite having a ‘position statement’ related to preprints,³¹ there is currently an ‘ethical vacuum’ pertaining to the ethics of preprint corrections and retractions that requires urgent debate, policy, and regulation, even more so now that lives are at stake during the COVID-19 pandemic.

However, many preprints are precursors of COPE member journals; in other words, many preprints end up being published in peer-reviewed COPE member journals. Therefore, the author is of the opinion that COPE has, through this association, the moral responsibility of adjudicating the ‘ethics’ pertaining to preprints, at least for its member journals and publishers, so as to maintain consistent ethical policies at the three main stages of the publication process: pre-publication (preprint), peer review, and post-publication peer review. The other important question that academics are surprisingly not asking is why none of the preprint servers (e.g. bioRxiv, medRxiv, COS's 26 preprint services, preprint.org, etc.) have applied to be COPE members. One would envision that publishing integrity, as represented nowadays by COPE membership (journal or publisher), with due scrutiny and approval by COPE before the attribution of membership, would represent an important objective of such preprint servers, which should perhaps, as the leading preprint platforms, set the example of the future of preprint-related publishing ethics.

Other concerns about preprints

The risks and concerns about the use and citation of preprints have, in fact, not changed much in the past few years, even though both the number and volume of preprint servers, and preprints, have been increasing (Johansson et al., 2018),³² with almost 50% of bioRxiv preprints being published in Elsevier, Nature, PLOS, and Oxford University Press journals (Anderson, 2020).³³ It is perhaps precisely because of their growth that concerns continue to exist. Given that they are citable items, and carry digital object identifiers (DOIs), involving a financial investment for DOI registration and the handling of processes related to preprint posting, online html text setting, indexing, and annual hosting costs, at least for COS preprint servers, supporters and owners of preprint servers understandably wish to promote them (even market them as brands) and their positive aspects in a ‘biased’ manner³⁴; they conveniently ignore the various risks and concerns raised in this paper and in others cited herein. The author considers the ‘promotion’ of preprints to be ‘biased’ because of an inherently skewed desire to ensure that invested efforts bear fruit. In this case,³⁴ Naomi Penfold, the first author, was the ASAPbio Associate Director in 2018–2020, while Jessica Polka, the second author, is the ASAPbio Executive Director.³⁵ In addition, PLOS, in which that paper was published, has a direct transfer agreement with bioRxiv and medRxiv, in the ‘Direct Transfer Program’ or B2J.³⁶ Some of the most pertinent risks include promoting false or misleading information³⁷ and a lack of clarity regarding policies between preprints and peer-reviewed journals.³⁸

The risks of preprints to human health and publishing integrity may have been amplified by their indexing on PubMed in a National Institutes of Health (NIH) pilot programme managed by the National Library of Medicine (NLM), but exclusively for NIH-supported research, running for a minimum of 12 months, since June 2020.³⁹ Allowing NIH-funded preprints to be indexed at PubMed, and its European affiliate Europe PMC, supported by high-ranking funding and health organisations (‘This COVID-19 preprints initiative is supported by a joint award from Wellcome, UK Medical Research Council (MRC), Swiss National Science Foundation (SNSF), and endorsed by the Chief Scientist of the World Health Organization (WHO)’),⁴⁰ has led to an explosion of results when the term ‘preprint’ is searched for at PubMed (2,767 total: 240 documents prior to 2020, 1,924 documents in 2020, and 603 documents in 2021).⁴¹ Currently, since keyword-based papers are not allowed to be distinguished from the manuscript type (preprint), preprints have now in essence received a stamp of ‘validation’, qualifying them as citable equivalents of peer-reviewed papers at PubMed. Although this programme is experimental, a previous experimental post-publication peer review ‘pilot programme’ at PubMed (PubMed Commons) turned out to be a failure (Teixeira da Silva, 2018b).⁴² The graphs at Europe PMC also show a rapid increase in the preprint ‘market’ by Research Square, two prominent clients being Springer Nature and Cambridge University Press.⁴³

There are thus currently three risks of, or problems associated with, allowing preprints to be hosted at, and indexed by, PubMed: 1) giving them equivalent intellectual and citation status as peer-reviewed papers; 2) allocating a ‘validation’ stamp that allows them to be used for medical and public purposes, including for COVID-19, despite not being peer-reviewed; 3) providing an unfair advantage to US-based research that is funded by the NIH, even though PubMed indexes global research with researchers from around the world who also publish work in preprints, a policy that some may perceive as unfair.

Conclusion

The current debate surrounding preprints is as fervent as it had been a few years ago. In fact, given the prominence of COVID-19 in global health, society, and academia, preprints have now vaulted to the top rank of discussion topics in academic publishing. As biological, medical, and other academic literature continues to evolve, especially the corrective aspects of that literature,⁴⁴ as preprints become more widely used, and as their volumes continue to rise, especially preprints related to COVID-19, the focus of the discussion is no longer ‘whether preprints should be cited’ but rather ‘how preprints should be cited’ to reduce the risks to the integrity of the literature and human health. Despite these concerns, to the author's knowledge, there is currently no tangible proof that a preprint (as a document or as a source of information) is a risk, or threat, to human health, or that more harm to human health has been caused by misinformation in a preprint than in a peer-reviewed journal. As much as peer-reviewed papers are supposed to be held to high standards, editors or managers of preprint servers should instil a basic, but rigorous, level of screening and quality control, at a minimum to establish open-science policies to ensure that basic ethical requirements are met.⁴⁵ Bonnechère recommends that, to avoid possible harm, a preprint be removed after the final paper is published in a peer-reviewed journal.⁴⁶ However, this is not advisable since a preprint represents a historical document that provides open and public evidence of the historical evolution of the paper, as well as a bibliometric record, and thus should never be removed. Ultimately, the integrity and credibility of preprints can be fortified only when they are treated as ‘ethical equals’ with peer-reviewed literature, for both COVID-19 literature⁴⁷ and other fields of study.

Source of funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflicts of interest

The author has no conflicts of interest to declare.

Ethical approval

The author confirms that this editorial had been prepared in accordance with COPE rules and regulations. Given the nature of the editorial, the IRB review was not required.

Authors' contributions

The author contributed solely to the intellectual discussion underlying this paper, literature exploration, writing, reviews and editing, and accepts responsibility for the content and interpretation.

Disclaimer

Despite the author’s generally critical or skeptical views about preprints, he recognizes the importance and prominence of this form of publication, and even has four preprints (one each) at preprint.org, ResearchGate, and OSF’s Psyarxiv and SocArXiv.