Abstract
This study describes the policies, submission requirements, and transparency in reporting and research integrity recommendations of academic preprint servers.
Preprint servers are online platforms that enable free sharing of preprints, scholarly manuscripts that have not been peer reviewed or published in a traditional publishing venue (eg, journal, conference proceeding, book). They facilitate faster dissemination of research, soliciting of feedback or collaborations, and establishing of priority of discoveries and ideas.1 However, they can also enable sharing of manuscripts that lack sufficient quality or methodological details necessary for research assessment, and can help spread unreliable and even fake information.2 Since 2010, more than 30 new preprint servers have emerged, yet research on preprint servers is still scarce.3 With the increase in the numbers of preprints and preprint servers, we explored servers’ policies, submission requirements, and transparency in reporting and research integrity recommendations, as the latter are often perceived as mechanisms by which academic rigor and trustworthiness are fostered and preserved.4
Methods
We conducted a cross-sectional analysis of, to the best of our knowledge, all known preprint servers that do not limit posting of manuscripts to authors with specific institutional affiliations or study funding (eg, Wellcome Open Research) nor actively seek out peer reviewers (eg, F1000) (see the eAppendix in the Supplement for server identification details). Between January 25 and March 31, 2020, M.M. analyzed servers’ web pages that resembled instructions to authors traditionally found in scholarly journals, as well as servers’ about, policy, and frequently asked questions pages. For each server, M.M. also went through the preprint submission process (without submitting a preprint) to check for additional information in the submission platforms (except for ChinaXiv, which required an email associated with a Chinese institution). Then, M.M. extracted data on explicit mentioning of 7 topics related to preprint policies, 6 to submission requirements, and 18 to transparency in reporting and research integrity that were deemed applicable across disciplines. The topics were informed by our previous analysis of journals’ instructions to authors and topics unique to preprints (see the eAppendix in the Supplement for details).5 On May 29, the number of records that servers hosted was documented, and on July 6, it was documented whether servers allowed health sciences discipline selection during the submission process and whether they hosted more than 500 such preprints (servers’ health sciences categories are listed in the data repository site).
Results
We analyzed 57 preprint servers that hosted approximately 3 million preprints in total. Of those, 10 servers hosted more than 500 health sciences preprints (Table 1). Of the 7 analyzed policies, the most commonly addressed across all servers was screening of preprints before or after they are made public (n = 47 [82%]) (Table 2). Two servers, Preprints.org and Research Square, used a screening checklist (the latter also provided a “badge” of passed checks). The most commonly addressed submission requirements were specifying the scholarly scope of preprints (n = 57 [100%]) and the study type allowed for deposit (n = 31 [54%]). Of the 18 analyzed recommendations on transparency in reporting and research integrity, preprint servers addressed a median of 1 recommendation (range, 0-11), most commonly data sharing (n = 22 [39%]). These recommendations were more prevalent (median, 5; range, 0-11) for the 10 servers with more than 500 health sciences preprints.
Table 1. Number of Records on Preprint Servers on May 29, 2020a.
| Server name | No. of recordsb |
|---|---|
| arXiv | 1 708 255 |
| Social Science Research Network (SSRN)c | 802 602 |
| EconStor (economics and business studies) | 119 864 |
| bioRxivc | 84 009 |
| RePEc/Munich Personal RePEc Archive | 49 164 |
| PhilArchive | 48 927 |
| Hyper Articles en Ligne (HAL) | 48 610 |
| ViXra | 35 827 |
| OSF Preprintsc | 17 174 |
| INA-Rxivc | 16 641 |
| Cryptology ePrint Archive | 14 817 |
| Preprints.orgc | 14 052 |
| ChinaXivc | 13 682 |
| Research Squarec,d | 12 962 |
| Mathematical Physics Preprint Archive | 9601 |
| PsyArXivc | 9475 |
| Social Science Open Access Repository (SSOAR) | 8193 |
| JMIR Preprintsc | 7888 |
| Optimization Online | 7531 |
| medRxivc | 5935 |
| SocArXiv | 5497 |
| LingBuzz | 5113 |
Servers not listed above include the following 12 servers with greater than 500 and less than 5000 records: ChemRxiv, Authorea Preprint Repository, PhilSci-Archive, Electronic Colloquium on Computational Complexity, Zenodo, EarthArXiv, LawArXiv, engrXiv, Thesis Commons, e-LIS, Earth and Space Science Open Archive (ESSOAr), and Advance (SAGEpub). Also not listed above are the following 23 servers with less than 500 records: EdArXiv, Commons Open Repository Exchange (CORE)/Humanities Commons, MarXiv, Arabixiv, AgriXiv, LIS Scholarship Archive (LISSA), EcoEvoRxiv, SportRxiv, MindRxiv, APSA Preprints, PaleorXiv, MetaArXiv, AfricArXiv, ECSarXiv, IndiaRxiv, FrenXiv, MediArXiv, NutriXiv, BodoArXiv, OARR: Open Anthropology Research Repository, FocUS Archive, MitoFit Preprint Archives, and BioHackrXiv.
The term records is intentional, as not all servers have filters that clearly differentiate between preprints and published articles or account for duplicate records.
These servers allowed health sciences discipline selection during the submission process.
For Research Square, only preprints not undergoing journal peer review were included.
Table 2. Preprint Servers’ Policies, Submission Recommendations, and Transparency in Reporting and Research Integrity Topicsa.
| No. (%) | ||
|---|---|---|
| All servers (n = 57) | Health sciences servers (n = 10)b | |
| Preprint policies | ||
| Screening check | 47 (82) | 8 (80) |
| Before a preprint is made public | 39 (68) | 6 (60) |
| After a preprint is made public | 8 (14) | 2 (20) |
| Authors advised to check preprint policies of journals | 40 (70) | 9 (90) |
| Commenting section for preprints provided | 39 (68) | 9 (90) |
| Versioning of preprints guidance provided | 30 (53) | 8 (80) |
| Instructions to authors page provided | 27 (47) | 5 (50) |
| Direct transfer of preprints to or from journals enabled | 10 (18) | 6 (60) |
| Text mining of preprints allowed | 7 (12) | 3 (30) |
| Submission guidance | ||
| Scope requirements | 57 (100) | 10 (100) |
| Specific (sub)discipline | 41 (72) | 3 (30) |
| All disciplines | 10 (18) | 5 (50) |
| All disciplines but for authors with region- or country-specific affiliation | 6 (11) | 2 (20) |
| Study type requirements (eg, experimental studies only) | 31 (54) | 6 (60) |
| Preprint structure recommended (eg, IMRaD) | 19 (33) | 7 (70) |
| Reference style recommended | 16 (28) | 5 (50) |
| Abstract guidance provided | 12 (21) | 5 (50) |
| (La)TeX format for submission allowedc | 10 (18) | 3 (30) |
| Transparency in reporting and research integrity | ||
| Data sharing | 22 (39) | 5 (50) |
| Recommended | 17 (30) | 4 (40) |
| Required | 4 (7) | 1 (10) |
| Linking or uploading data allowed | 1 (2) | 0 |
| Plagiarism addressed | 15 (26) | 4 (40) |
| ORCID ID recommended | 14 (25) | 5 (50) |
| Errata guidance provided | 12 (21) | 4 (40) |
| Conflicts of interest declaration required | 9 (16) | 4 (40) |
| Ethics approval declaration required | 9 (16) | 6 (60) |
| Funding declaration required | 9 (16) | 4 (40) |
| Authorship guidance provided | 8 (14) | 3 (30) |
| Null or negative results studies invited | 6 (11) | 0 |
| ICMJE recommendations endorsed | 5 (9) | 2 (20) |
| Patenting addressed in relation to preprints | 4 (7) | 2 (20) |
| Replication studies invited | 3 (5) | 1 (10) |
| Reporting guidelines recommended | 3 (5) | 2 (20) |
| COPE recommendations endorsed | 2 (4) | 1 (10) |
| Image manipulation addressed | 2 (4) | 1 (10) |
| Study limitations reporting required | 2 (4) | 2 (20) |
| TOP guidelines endorsed | 2 (4) | 1 (10) |
| Statistical reporting guidance provided | 0 | 0 |
Abbreviations: COPE, Committee on Publication Ethics; ICMJE, International Committee of Medical Journal Editors; IMRaD, Introduction, Methods, Results, and Discussion; ORCID ID, Open Researcher and Contributor ID; TOP, Transparency and Openness Promotion Guidelines.
Topics are listed in order of frequency mentioned on websites.
Ten servers allowed health sciences discipline selection during the submission process and hosted more than 500 such preprints on May 29, 2020: bioRxiv, ChinaXiv, INA-Rxiv, JMIR Preprints, medRxiv, OSF Preprints, Preprints.org, PsyArXiv, Research Square, and Social Science Research Network.
(La)TeX is a text markup system often used in academia as an alternative to direct formatting systems (eg, Word or Pages).
Discussion
Although most preprint servers used screening checks for preprints, they provided little explicit guidance on issues that are important for transparency in reporting and research integrity. Disciplinary differences observed for such recommendations in journals5 were also present for preprint servers, with more recommendations addressed by servers hosting more than 500 health sciences preprints. The study limitations include data extraction by 1 author, that analyzed topics were not comprehensive, and that many topics were more prominently discussed and therefore may be more commonly addressed in the biomedical literature. Also, servers may follow policies and scholarly standards that are not explicitly mentioned on their websites. Nevertheless, there is an opportunity for servers to encourage and require transparent reporting of research, adherence to research integrity standards, and detailed statements of policies and submission requirements. In doing so, they could improve quality and trust in scholarly information exchange.
Section Editor: Jody W. Zylke, MD, Deputy Editor.
eAppendix. Preprint Servers Identification and Topic Selection Process
eReferences
References
- 1.Chiarelli A, Johnson R, Richens E, Pinfield S Accelerating scholarly communication: the transformative role of preprints. Zenodo. Published online September 29, 2020. doi: 10.5281/zenodo.3357727 [DOI]
- 2.Ferrara E. What types of COVID-19 conspiracies are populated by Twitter bots? arXiv Preprint posted April 20, 2020. doi: 10.5210/fm.v25i6.10633 [DOI]
- 3.Rittman M. Preprint servers. Google Docs. Accessed Aug 12, 2020. https://docs.google.com/spreadsheets/d/17RgfuQcGJHKSsSJwZZn0oiXAnimZu2sZsWp8Z6ZaYYo/edit#gid=0
- 4.Resnik DB, Master Z. Policies and initiatives aimed at addressing research misconduct in high-income countries. PLoS Med. 2013;10(3):e1001406. doi: 10.1371/journal.pmed.1001406 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Malički M, Aalbersberg IJ, Bouter L, ter Riet G. Journals’ instructions to authors: a cross-sectional study across scientific disciplines. PLoS One. 2019;14(9):e0222157. doi: 10.1371/journal.pone.0222157 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix. Preprint Servers Identification and Topic Selection Process
eReferences