Abstract
Introduction
Complex foot and ankle trauma surgery is a super-specialization of orthopedic trauma surgery. This topic is gaining in interest, both clinically and in research. The peer review process is the cornerstone of the current scientific model of evaluation of research papers. The burden of peer review activity in foot and ankle trauma surgery is currently unknown.
Material and method
A comprehensive Microsoft Outlook® analysis of over seven years’ worth of review invites from one surgical Foot & Ankle specialist working at a Level-1 trauma center was evaluated. The topic of each paper was recorded. Whether or not the journal was specifically foot ankle, orthopedic, general medicine or predatory of origin was noted. In addition, it was recorded if the journal was indexed in Pubmed, Medline, or Scopus. The topic of the paper was scored as being foot and ankle related, and more specifically foot and ankle trauma related. Review of revisions were scored as invites as well.
Results
From January 2013 to December 2019 a total of 467 review invites were received. The number of peer review invitations rose from 14 in 2013 to 127 in 2019. The percentage of foot and ankle specific journals was 27.4%. The number of requests from non-indexed (predatory) journals rose from 3.7% in 2014 to 36.2% in 2019, with a decrease in article topics related to the authors subspecialty. The annual absolute number of performed reviews stayed approximately the same throughout the study period (34 on average in the last 5 years).
Conclusion
The number of review requests in foot and ankle surgery has increased about 5-fold over the 5 year period evaluated. There is a need for more reviewers to spread the burden. Therefore, foot and ankle specialists who are prepared to publish, should also be prepared to provide peer reviews to maintain a high level of quality in foot and ankle research.
Keywords: Peer review, Foot and ankle, Trauma, Predatory, Journal
Zusammenfassung
Einleitung
Komplexe Fuß- und Sprunggelenktraumatologie ist ein Sub-Spezialgebiet in der Orthopädie und Unfallchirurgie. Sie findet zunehmend auch in der Forschung Interesse. Der Begutachtungsprozess durch Fachexperten (Peer Review) ist die Basis des derzeitigen Modelles der Evaluierung wissenschaftlicher Arbeiten. Derzeit ist unbekannt, wie sich die Belastung durch den Peer Review für den einzelnen Wissenschaftler darstellt.
Material und Methoden
Eine gründliche Analyse in Microsoft Outlook® von Anfragen zur Begutachtung an einen Fuß- und Sprunggelenkspezialisten an einem Level-1 Traumazentrum wurde über einen 7-Jahreszeitraum durchgeführt. Das Thema jedes Beitrages wurde erfasst. Die anfragende Zeitschrift wurde in speziell für Fuß und Sprunggelenk, Orthopädie/Traumatologie, Medizin allgemein oder ohne Zusammenhang bzw. von Räuberzeitschriften stammend klassifiziert. Zusätzlich wurde die Indizierung in Pubmed Medline oder Scopus erfasst. Das Thema der zu begutachtenden Arbeiten wurde in Fuß und Sprunggelenk-assoziiert und spezifisch Fuß- und Sprunggelenktrauma eingeteilt. Die Begutachtung von Revisionen wurde ebenfalls als volle Begutachtung gewertet.
Ergebnisse
Zwischen Januar 2013 und Dezember 2019 wurden 467 Einladungen zur Begutachtung registriert. Die Anzahl stieg von 14 im Jahre 2013 auf 127 im Jahre 2019. Der Anteil der nichtindizierten (Räuber-) Zeitschriften stieg von 3.7% im Jahre 2014 auf 36.2% im Jahre 2019, wobei die Anzahl der spezifischen Anfragen für das Spezialgebiet des Autors sank. Die absolute Zahl an Begutachtungen blieb über die vergangenen 5 Jahre mit durchschnittlich 34 pro Jahr in etwa gleich.
Schlussfolgerung
Die Anzahl von Begutachtungsanfragen in der Fuß- und Sprunggelenkchirurgie hat in den vergangenen Jahren um etwa das 5-fache zugenommen. Hierdurch entsteht der Bedarf, die Last gleichmäßiger zu verteilen. Fuß- und Sprunggelenkspezialisten, welche selber publizieren, sollten daher auch bereit zum Peer Review-Prozess sein, um eine hohe Qualität der Forschung in unserem Fachgebiet zu gewährleisten.
Schlüsselwörter: Begutachtung, Fuß und Sprunggelenkchirurgie, Trauma, Räuberzeitschriften, Fachzeitschrift
Introduction
All involved in scientific endeavors are well aware of the peer review process. Following the submission of a manuscript to a certified journal the (section) editor decides if the standards of the journal are met, and if so, invites at least two peers to comment on the manuscript.
This process, also known as refereeing, is the process of subjecting an author's scholarly work, research, or ideas to critical appraisal by others who are experts in the same field. The peer review process aims to improve the quality of the paper and the research it contains. In addition, through peer review manuscripts of poor or substandard quality are expectedly filtered out. Ideally, this process ensures reliable data that could be implemented in the reader's own daily practice. In addition, many authors feel their paper benefitted from the peer review process. By reviewing high quality manuscripts, the peer reviewers can enhance their own personal knowledge, creating a win-win situation [1], [2], [3].
There is much to be said about the benefits and drawbacks of the peer review process [4], [5]. It is estimated that there are over 28,000 scientific journals worldwide, which publish 2.5 million scientific papers annually [6], [7]. Each paper submitted needs peer reviewing, usually by at least two reviewers [2], [3]. The real number of peer reviews is actually much higher than 2.5 million, as this number does not include the rejected and resubmitted papers or review of revisions of an eventually accepted paper.
The burden of peer review has been the topic of several papers, showing that most of the reviews are performed by a minority of the researchers [7].
In Medline the annual number of references has doubled almost every 10 years, with over 1 million references in 2015 alone [8]. With the rise of open-access the number of research papers has increased even further. A thorough review of a single paper can take several hours.
While being asked to review may be considered an honor, it may also be seen as a burden [3], [4], [9]. Previously the rapid increase between 2013 and 2018 in journal and congress requests from non-indexed journals was reported, with an increase from approximately 5% to 41% of all received junk mail from 1 academic scholar [10], [11]. In 2019 the number of emails concerning predatory journal and congress invitations as a percentage of all received junk mail has risen even further to 56%.
The aim of the current study was to 1) quantify the increase in review requests in a specific sub-specialty (Foot and Ankle Trauma Surgery), 2) calculate the magnitude of predatory journals in review requests, 3) evaluate the accuracy of invites per journal type, and 4) see if there is reviewer's fatigue.
Material and method
All review requests sent per email were collected from one surgical Foot & Ankle specialist working at a Level-1 trauma center (TS) starting from one year after opening the current work-related email address. All email invites from July 1st, 2013 through December 31st, 2019 were evaluated. The results were analyzed per year.
A comprehensive Microsoft Outlook® analysis of almost seven years’ worth of review invites was evaluated. The topic of each paper was recorded. Journals were classified as either specifically foot ankle (including both orthopedic and/or trauma related), general medicine, or predatory. Predatory included all journals that were not indexed in Pubmed, Embase or Scopus. The topic of the paper was assessed to categorize if it was foot ankle related, and more specifically foot and ankle trauma related, which is the subspecialty of the author. Review of revisions were scored as invites as well.
Results
From July 1st 2013 to December 31st 2019 a total of 467 review invites were received. The number rose from 20 in 2014 to 127 in 2019, which represents a 5-fold increase in review invitations (Figure 1 ). A total of 75 (16.1%) requests were for re-reviews. Even though the percentage performed reviews appears to decline, the absolute number of performed reviews stayed roughly the same throughout the years (Figure 1). The average number of reviews completed by the author per year over the last five years was 34 per year.
Figure 1.
Number of peer review requests and performed reviews per year.
The percentage of foot and ankle specific journals was 27.4%. The percentage of requests per year from non-indexed (predatory) journals rose from 3.7% in 2014 to 36.2% in 2019.
Overall 362 requests were foot and ankle related, and 254 were foot and ankle trauma related. A total number of 134 invites came from non-indexed journals, 128 of which were classified as coming from “predatory journals”, which were all rejected.
Requests within the specialty of the author were more likely to be reviewed. The number of requests specific to the specialization of the reviewer is shown in Table 1 . The non-indexed journals performed significantly lower compared to others regarding their aim of the request. The quality and specificity of the review request is also reflected by the number and percentage of actually performed reviews. While approximately 70% of the requested reviews from the field of foot and ankle surgery were performed, only 4% of the reviews requested by non-indexed journals were actually performed. Table 2 provides an overview of the non-specificity of review requests sent by some of the predatory journals.
Table 1.
Percentage of foot and ankle related requests and number of performed reviews per type of journal.
| Type of journal | Total number of requests | Foot and ankle related request | Foot and ankle trauma related requests | Review performed |
|---|---|---|---|---|
| Foot and ankle specific | 128 | 128 (100%) | 91 (71%) | 92 (72%) |
| Orthopaedic and Trauma surgery |
161 | 139 (86%) | 95 (59%) | 82 (51%) |
| General medicine | 44 | 31 (70%) | 28 (64%) | 19 (43%) |
| Non-indexed | 134 | 64 (48%) | 40 (30%) | 6 (4%) |
Table 2.
Topics of interesting “non-peer review” requests from predatory journals.
| 2020 | Treatment of cubitus varus in children |
| 2020 | Experimental Gynecology |
| 2020 | Subcutaneous cysts with wriggles inside |
| 2020 | Genetic Differences Between Two Species Of Morphologically Identical Metapenaeus Genus |
| 2020 | Fatigue Fracture of Double-Edge-Cracked Ti/APC-2 Nanocomposite Laminates |
| 2020 | Bio briquetting |
| 2020 | Serum Cardiac Markers Coronavirus Disease |
| 2019 | Hemothorax in Hemophiliac B |
| 2019 | The Consciousness Energy Healing Treatment |
| 2019 | Hypertrichosis in a Newborn Infant |
| 2019 | Foot reflexotherapy |
| 2019 | Cyanotic congenital cardiopathies |
| 2019 | Ofuji disease |
| 2018 | Metaplastic Breast Carcinoma |
| 2018 | Infrared sintering of gears |
Discussion
The main findings of the current study can be summarized as follows. 1) There is a substantial and constant increase in review requests in a specific sub-specialty (foot and ankle trauma surgery) over the last 5 years. 2) Only about every fourth request (27.4%) came from foot and ankle specific journals. 3) The percentage of requests per year from nonindexed (predatory) journals increased ten-fold over the observation period. 4) While the percentage of actually performed reviews declined over time, the absolute number of reviews per year remained constant.
Thus, although in this particular study there is no evidence for reviewer's fatigue, the increasing number of reviews leads to a higher rate of rejection of review requests. Due to the increase of requests a selection has to be made which articles to review. Usually these are the higher indexed journals in the field of the (academic) reviewer. This may lead to less rigorous reviewing of papers from lower indexed journals. As the total number of submitted and published manuscripts is steadily on the rise (the number of peer review requests in the first 20 weeks of 2020 is already 52), to meet the increased needs of the peer review process the number of scientists willing to assist in the task of reviewing should also increase. Ideally, both the authors and the reviewer will benefit from and learn during the review process. In contrast, reviewing manuscripts for obvious predatory journals will unlikely lead to improved higher quality papers, given the fact that most review requests (70%) were not related to the subspecialty of the author, indicating a non-peer reviewing process. This mis-targeting appears to be getting worse over time.
Between 2014 and 2017 a total of 49 predatory review requests were received of which 33 were foot and ankle related and 21 (43%) were foot and ankle trauma related. In 2018 and 2019 a total of 83 predatory review requests were received, of which 29 were foot and ankle related and 18 (22%) were foot and ankle trauma related (Fisher exact test: p = 0.0174).
These “predatory review requests” are typically easy to identify. They start with an automatically generated greeting with abbreviated names that are frequently misspelt, followed by standard sentence like “season's greetings”, “hope you are doing well” and a review request in simple to poor English. The deadline is typically very short (about one week) to meet the quick time to publication as promised for the authors’ money. Also, in the majority of cases, the topic of the manuscript will not match the scientist's expertise, as demonstrated by the current study. Some of the most striking examples that reached the authors’ junk mail folders over the last few years are summarized in Table 2.
Interestingly, the number of invitations between July 1st 2013 and December 31st 2018 to submit an article to predatory journals for the same author (TS) was more than 7,000 [11]. This number was about 50 times higher than the number of review requests from this group of journals. The same tendency was noted by the second author of this study (SR) over the last year. This discrepancy between invitations to submit manuscripts and invitations to review indicates that the review process in these journals is not as “rigorous” as stated in most of the requesting emails. This may be another clue that paper quantity (i.e. money), not scientific quality, is the main focus in these journals [11], [12]. This raises serious doubt on the claim of many of these predatory journals of being peer reviewed.
As the number of published papers about foot and ankle surgery has increased substantially over the last few years, so did the number of review requests in this field. In order to maintain a high level of quality with a minimum of two expert reviewers per manuscript, there is a true need for more reviewers in order to spread the burden more evenly. Therefore, foot and ankle specialists who are prepared to publish, should carefully select the target journal [11]. They should also be prepared to review manuscripts from others only in indexed journals to maintain a high level of quality in foot and ankle research.
Ethical Approval
No ethical approval was needed for this study
Funding
No funding was received
Conflict Of Interest
The authors declare no conflict of interest
References
- 1.Malay D.S. So, why peer review? J Foot Ankle Surg. 2008;47(4):265–266. doi: 10.1053/j.jfas.2008.05.002. [DOI] [PubMed] [Google Scholar]
- 2.Pinzur M.S. The Peer Review Process. Foot Ankle Int. 2018;39(11):1383–1384. doi: 10.1177/1071100718797614. [DOI] [PubMed] [Google Scholar]
- 3.Twaij H., Oussedik S., Hoffmeyer P. Peer review. Bone Joint J. 2014;96–B(4):436–441. doi: 10.1302/0301-620X.96B4.33041. [DOI] [PubMed] [Google Scholar]
- 4.Iantorno S.E., Andras L.M., Skaggs D.L. Variability of Reviewers’ Comments in the Peer Review Process for Orthopaedic Research. Spine Deform. 2016;4(4):268–271. doi: 10.1016/j.jspd.2016.01.004. [DOI] [PubMed] [Google Scholar]
- 5.Malay D.S. Peer review and scientific misconduct: bad authors and trusting reviewers. J Foot Ankle Surg. 2009;48(3):283–284. doi: 10.1053/j.jfas.2009.03.001. [DOI] [PubMed] [Google Scholar]
- 6.Huisman J., Smits J. Duration and quality of the peer review process: the author's perspective. Scientometrics. 2017;113(1):633–650. doi: 10.1007/s11192-017-2310-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Kovanis M., Porcher R., Ravaud P., Trinquart L. The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise. PLoS One. 2016;11(11):e0166387. doi: 10.1371/journal.pone.0166387. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Laakso M., Welling P., Bukvova H., Nyman L., Bjork B.C., Hedlund T. The development of open access journal publishing from 1993 to 2009. PLoS One. 2011;6(6):e20961. doi: 10.1371/journal.pone.0020961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Sylvia L.M., Herbel J.L. Manuscript peer review--a guide for health care professionals. Pharmacotherapy. 2001;21(4):395–404. doi: 10.1592/phco.21.5.395.34493. [DOI] [PubMed] [Google Scholar]
- 10.Schepers T., Rammelt S. Dear Dr.%%Firstname%%: Predatory journals (Räuberzeitschriften) in Foot and Ankle Surgery: Editorial. Fuss und Sprunggelenk. 2019;17(2):59–60. [Google Scholar]
- 11.Schepers T., Rammelt S. The burden of predatory journal and congress requests in foot and ankle surgery. Fuss und Sprunggelenk. 2019;17(2):61–67. [Google Scholar]
- 12.Rammelt S. More than just another spam: Predatory emails to Foot & Ankle Surgeons. Foot Ankle Surg. 2019;25(5):551–552. doi: 10.1016/j.fas.2019.08.013. [DOI] [PubMed] [Google Scholar]