Abstract
Purpose
This study’s primary objective is to evaluate the time spent in the peer-review process for orthopaedic publications related to the Covid-19 pandemic. The secondary objective is to evaluate the countries’ and journals’ contributions in these publications.
Materials and methods
We carried a search on August 1st, 2020, in one database (PubMed) using limited search terms to identify the orthopaedic publications related to the Covid-19 pandemic. After a filtration process, we evaluated the eligible article to identify the prevalence of different articles types, different countries, and journal contributions in these publications. The evaluation of the time spent in the peer-review process was done by obtaining the submission and acceptance dates.
Results
Of the 231 articles eligible for initial assessment, review articles were the most common article type published (51%), 48 countries published articles in a unique 78 journals. Evaluation of the peer-review process in 147 articles revealed that the mean time from submission to acceptance was 14.3 ± 15.8 days (range from 0 to 74), the peer-review process took less than 30 days in 127 (86.4%) articles, 15 (10.2%) articles were accepted within the first day of submission.
Conclusion
Orthopaedic community contributed significantly to the publications related to the Covid-19 pandemic, with a contribution from many countries and journals. The peer-review process was notably shortened for some articles.
Keywords: Covid-19, Orthopaedics, Peer-review, Infodemic
1. Introduction
Since the declaration of Covid-19 as a “Public Health Emergency of International Concern” by the World Health Organization (WHO) on January 30th, 2020, followed by announcing it as being a pandemic on March 11th, 2020, a surge of publications in various scientific aspects on Covid-19 disease by different journals has taken off in the literature.1, 2, 3, 4
Gazendam et al. used the term “infodemic” to describe the “plethora” of publications related to the Covid-19 pandemic, The WHO warned about this “infodemic” phenomenon concerning news and social media; however, it seems that this phenomenon found its way to the scientific writing as well.3
Kambhampati SBS et al. studied the expansion of medical literature publication numbers on Covid-19 in April 2020 and found that the average number of daily publications ranges from 34.8 to 58.9 articles per day.2 In another study by Yeo-Teh and Tang carried in June 2020, they found that the average rate was 137 papers per day.1
The most concerning issue of the rush and racing in scientific publications on the newly experienced pandemic resides in the potential for “misinformation” or “disinformation,” which may be caused by expressing false or misleading claims combined with a deficiency in the reviewing process, which is considered as the information checkpoint before it gets published to the public.3,5 This concern was confirmed in a study by Yeo-Teh and Tang where they found that the retraction rate of Covid-19 related articles (since the start of the disease) exceeds the rates of cancer and immunology related articles retraction rates, a prominent example was the article written on the role of antimalarial drugs in the management of Covid-19 patients which was published in The Lancet on May 22nd and retracted on June 4th. They mentioned that one of the possible causes of an increased rate of retraction is the “shallowness” of the peer review process.1,6
The primary objective of the current study was to evaluate the time taken in the peer-review process for orthopaedic publications related to the Covid-19 pandemic. The secondary objective was to evaluate the prevalence of article types, the countries, and journals’ contributions to these publications.
2. Materials and Methods
2.1. Search strategy and selection criteria
We performed an electronic search on the PubMed database (as it is considered as one of the most reliable and commonly used free search engine giving access to the MEDLINE database) on August 1st, 2020 covering the period from December 1st, 2019, to retrieve all potential orthopaedic articles published related to Covid-19 pandemic (regardless the journal specialty in which the article was published). The following search terms were used [((orthopaedic) OR (orthopaedic)) AND (Covid)], there was no search filtering applied regarding language and study design.
Results from the scanned database were downloaded as an excel sheet (which contained all the articles’ primary data of interest) and were grouped as well into one Endnote library. After finding and excluding duplications using the endnote software, duplicates were removed from the excel sheet.
The filtration process to retrieve the eligible articles was carried in steps; the first filtration step was based on screening the titles, the two authors independently reviewed all the titles of the articles in the excel sheet, and we agreed to include all titles in which a clear relation between Covid-19 pandemic and any field of Orthopaedics (including all subspecialties: Trauma, Sport, Hand, Spine, etc.) regardless the specialty of the journal where it was published, and to exclude articles concerned with Covid-19 but not related to Orthopaedics (even if it was published in an Orthopaedic specialized journal). After the agreement of both authors on the initial filtration by titles, we examined the full manuscript of the remaining articles to obtain further data to carry further filtration steps (if a non-English article was found, we used the google translation service to be able to retrieve the desired data from the article). The second filtration step was excluding articles according to article type; the following article types were excluded from being involved in the final analysis: Annotation, Commentary, Communication, Editorial, Letter to Editor, and Reply to letter. We classified the remaining included articles (according to the article type reported by the journal or by examining the methodology section of the article) into systematic reviews, review articles, original articles (which was either observational or analytical studies including patient data and implementing mathematical and statistical modeling), consensus articles (in which the data in the article was based on a consensus meeting), survey (in which the article was based on a questionnaire or a survey), case reports and technical notes. After this step, we could identify the overall countries contribution based on the nationality of the institution to which the authors are affiliated; we identified the percentage of articles originated from one country and the articles published as a collaboration between institutions affiliated to different countries (multi-national publications), we classified the form of contribution into three categories: Ⅰ-country published only by their local institutions, Ⅱ-country published only as a contribution in a multi-national publications and Ⅲ- countries which combine both Ⅰ and Ⅱ category. Journals contributions were determined at this step by ordering the journals according to the numbers of publications.
The final filtration step was excluding articles that lack reporting the submission and/or the acceptance dates; after this step, we could identify the articles eligible for evaluating the peer review process by calculating the time taken from submission to acceptance as well as reporting any incidence of required revision.
Descriptive statistics were reported as the mean and standard deviation, numbers, and percentages. Median and range were reported when mentioning were beneficial.
3. Results
The study selection procedure and articles filtration process are summarized in the flow diagram (Fig. 1A). After evaluating 231 articles, review articles were the most published, constituting 51% of the total publications (Fig. 1B). Countries’ contribution was as follows: A total of 48 countries contributed to Covid-19 related orthopaedic publications. Articles by authors affiliated to institutions from the same country were 199 (86.1%), and 32 (13.9%) articles (multi-national articles) were a co-authorship between authors affiliated to institutions from at least two different countries. Nine (18.8%) Countries in category Ⅰ, 13 (27%) countries in category Ⅱ, and 26 (54.2%) countries in category Ⅲ (Fig. 2A). As for the journals involved, we identified 78 unique journals (including 49 (62.8%) Orthopaedics specialized journals and 29 (37.2%) other different medical specialty journals), the top 10 journals (all are orthopaedic specialized) published 125 (54.1%) of the total articles. The top 10 journals have different nationalities and represented various societies and associations from different countries; four were multi-national journals (International Orthopaedics, J orthopaedic trauma, Injury, and Acta Orthopaedica), three American (J Bone Joint Surg Am, J Arthroplasty, and J Am Acad Orthop Surg), two Indian (J Clinical Orthopaedics and Trauma and J Orthopaedics), and one journal from Iran (Archives of Bone and Joint Surgery) (Fig. 2B).
Fig. 1.
A, The flowchart shows the PubMed database search results and the articles identified at each filtration step, with the reasons for exclusion. B, Distribution of article types.
Fig. 2.
A, Publications distribution according to the nationality of the institutions to which the authors are affiliated, map chart showing all countries’ contributions (+category Ⅰ, ∗ category Ⅱ, and +∗ category Ⅲ), a bar chart showing the top 10 contributing countries. B, Top 10 journals (according to publication numbers), the time spent (in days) in the peer-review process for articles published in these journals is included, it is not applicable (N/A) to include the peer-review data for journals which did not report the submission or/and acceptance dates. C, Details of the peer-review process for a total of 147 articles showing time spent from submission to acceptance and the peer-review process for articles (number and percentage) after grouping according to weeks (wks) taken for acceptance.
Peer-review process evaluation revealed that of the 231 articles, the dates of submission and acceptance were reported in 147 (63.6%), of the excluded 84 articles, one article from Canada7 the journal stated that the article was not peer-reviewed. In 83 articles, the data on submission and/or acceptance was not reported. Analysis of the peer-review cycle for the eligible 147 (100%) articles showed that the mean time (in days) from submission to acceptance was 14.3 ± 15.8 days (median was 9, the range was from 0 to 74) where “0 days” means that the article was accepted at the same day of submission. In 47 (32%) articles, acceptance was after at least one revision, the mean time (in days) between submission and revision was 11.5 ± 16.3 (median was 3, the range was from 0 to 69), the mean time between revision and acceptance was 2.4 ± 4.2 (median was 1, the range was from 0 to 20). The longest article in the peer-review process (spent 74 days from submission to acceptance) was from China8; however, most of the time (69 days) was spent during the revision process (which may give the impression that the delay was from the authors’ side). In 127 (86.4%) articles, the review process took 30 days or less, 15 (10.2%) of the 147 articles were accepted within the first day of submission, 29 (61.7%) of the 47 revised articles were accepted within the first day of submitting the revised version. For the top 10 publishing journals, the peer-review process could be evaluated in six journals (four journals did not report the submission and/or the acceptance dates), the mean time spent in the peer-review process in these journals was 16.5 ± 10.8 days, ranging from 0 to 44, which was relatively less than the mean time reported for all journals. Details of the peer-review process results for all the articles and articles published in the top 10 journals are shown in (Fig. 2B and C).
4. Discussion
The fast spread of the pandemic among different countries seemed to have its implication on social media as well as medical journals.3 As many journals rushed into publishing articles related to Covid-19, offering a fast track peer review process for articles on this subject and giving free access to articles published. Orthopaedic surgeons and orthopaedic related journals found its way in this regard.9
The main findings of the current study are that review articles were the most published from the orthopaedic community concerning Covid-19, and interestingly, the peer-review process for some articles was accelerated to take less than one day.
Gazendam et al. carried a PubMed search between December 1st, 2019, to March 31st, 2020, to evaluate different characteristics of Covid-19 related scientific publications; their search revealed 3212 records.3 Kambhampati SBS et al. carried a PubMed search (using different search terms) on the same subject as in the previous study in April 2020, which revealed 6831 articles2; by the time of the current manuscript preparation in August 2020, we used the same search strategy used by Kambhampati SBS et al., the search revealed 40,843 articles, this means that the Covid-19 related publications increased by nearly six folds in four months.
Before applying the filtration process in the current study, we defined 329 articles by specialized orthopaedic journals. In contrast, Kambhampati SBS et al.2 at the time of their study, they defined only 35, which means that the orthopaedic journals increased their production of articles (any article type) related to Covid-19 by about nine folds. Abdelnasser MK et al.10 performed a PubMed and Scopus databases search on May 22nd to identify the Orthopaedics publications related to Covid-19 (the same concern in the current study). However, they used more precise search terms than what we used in the current study; they retrieved 262 articles compared to 231 articles in the current study.
Gazendam et al., after studying 1741 articles, found that the studies originated from 59 countries and were published across 447 unique journals (24 (5.4%) were surgical journals). China came as the lead country in publications counts, followed by the United States and the United Kingdom.3 Although our study concentrated on the orthopaedic related publications, we noticed the same diversity noticed by the previous study, with 48 countries contributing to the publications with the United States representing the leading country, followed by Italy and India. The publications were published in 78 unique journals, the low number of journals compared to the previous study explained by the selectivity (one specialty) in the search terms in the current study.
Regarding the most publishing orthopaedic journals, Kambhampati SBS et al. found that the Journal of Bone and Joint Surgery (JBJS) American was the most publishing journal by the time of their study,2 which was the same finding in our study; however, the order of the rest of top ten publishing journals was different in our study as shown in (Fig. 2B). Six of the top publishing journals originated from the top 10 contributing countries (USA, India, and Iran).
In the current study, we found that review articles represented the majority of published articles (51%); the same finding was reported by Gazendam et al. where they found that the most common publication types were commentaries and narrative reviews, forming about (63.4%) of all publications, while original articles accounted for (30.6%).3 The same finding was reported as well in the study by Kambhampati SBS et al.2 The trend that reviews were the most published compared to original articles can be explained by the novelty of the condition with a limited time to design, execute, and publish a clinically driven article.3
The essence of the rigorous peer-review process is to catch the misconduct and shortcomings of scientific articles before and not after it being published; it is considered as the foundation for the integrity of scientific findings in the medical field.11,12 Although this process had been practiced by most of the medical journals over decades, however, the debate regarding the integrity and quality of this process still exists.13 This process may be affected by some factors, especially those related to the reviewers, if they are young with less experience, lacking skills, and had no enough training on properly performing this process.12
Gazendam et al. found that the mean time from submission to publication was 13 ± 12 days (median 10 and range, 0–113 days), 14.9% of the articles were published within seven days of submission.3 We reported nearly the same trend in orthopaedic publications related to Covid-19, where the mean time from submission to acceptance was 14.3 ± 15.8 days (median 9, the range was from 0 to 74), and 44% of the articles were accepted within the first week.
A breach in the peer-review process was suggested as a factor contributing to the alarmingly high number of retractions reported in the study by Yeo-Teh and Tang.1 They found 17 retracted articles on Covid-19 after performing a search on the Retraction Watch database (RWD) (https://retractionwatch.com/retracted-coronavirus-covid-19-papers/) on June 8th, by the time of preparing this article we did the same search using the same search terms they used; however, it revealed 32 retracted articles, which means that the number nearly doubled in less than three months. Although the paper on Covid-19 management using antimalarial drugs published in the Lancet had been retracted shortly after its publication,6 surprisingly, this paper was cited about 68 times, most of the citations occurred before it was retracted, which may have an adverse effect on the therapeutic interventions and approaches based on the weak data and inaccurate results reported in the retracted article, with a negative effect on the health systems and patients.1
The article processing journey in most journals starts shortly after the submission; as it is first checked by a Publishing Assistant to check the initial appropriateness of this article to the journal, then it is passed to the Editor, who decide if this article is within the journal scope and will add to the specific field knowledge, if passed this stage, it will be sent to be “peer-reviewed” with different levels of review process blindness.14,15 How much time the article will spend in the review process depends on the reviewer’s expertise, organization, and the available time. After this, a final decision is made regarding the article, which, if rejected it will go through the same process in another journal, and if a revision was recommended, it might go through the same process in the same journal after submitting the revised version.14
The surge in the number of publications on the Covid-19 pandemic can be explained by the novelty and seriousness of the conditions, which pushes the researchers in different specialties to suggest guidelines and solutions for the proper way to deal with this new disease. So, what happened to the peer-review process during this pandemic? Leading to its notable shortening in some articles. First, the overflow of submissions surely put the peer review system under immense stress; however qualified researchers (who are the best for performing the review process) are limited besides being busy preparing their research, this may have forced the journals to use the help of young less experienced reviewers to cope with the increasing number of submissions. Second, the competence between journals on publishing on this new subject may also be a driving force behind shortening the peer review cycle time.1 Third, most surgeons suspended their elective work which gave them more time to put in the review process and also gain knowledge about the condition. Fourth, the push by the journal editorial offices on all who are involved in the peer-review process to shorten the process as possible. Lastly, the change in the policies of some journals such as accelerated processing, offering fee waivers, and promising a fast track peer review process may also have played as a contributing factor.1
The current study has some limitations. First, we only did a “superficial” search using only one database and a limited search keyword, which may lead to missing some articles related to the subject in concern. Second, during the filtration process, we decided to included articles by initial screening of the titles to check if it is related to Orthopaedics or not which will give the idea about orthopaedic surgeons production and not of the orthopaedic journals as in the way of filtration we skipped many articles published in orthopedic journals which were not related to the orthopaedic practice. Third, of the top 10 contributing journals, four did not report data regarding the review process, which if present, may have its effect on the results. Last, the examination and selection were only carried by the two authors, which may lead to bias and mistakes; however, we overcame this problem by double-checking the data at each filtration step.
5. Conclusion
There is a rise in the article’s production from the orthopaedic community concerning the Covid-19 pandemic (mostly review articles), with a contribution and collaboration from different countries in a vast number of journals. The peer-review process was notably shortened for some articles.
Funding
There is no funding source.
Consent for publication
All authors approved the final manuscript and its appropriateness for submission and publication.
Ethics approval
N/A.
Consent to participate
N/A.
Availability of data and material
All relevant data are included in the manuscript.
Authors’ contribution
Both authors contributed equally to all the aspect of the manuscript.
Declaration of competing interest
The authors declare that they have no conflict of interest.
Acknowledgment
We would like to express our great thanks and appreciation for all the personnel involved in the peer-review process who pay from their time and effort to fasten the publication of scientific articles, with an especial appreciation for the reviewers.
Contributor Information
Ahmed A. Khalifa, Email: ahmed_adel0391@med.svu.edu.eg.
Ahmed M. Ahmed, Email: drahmedabdou1993@gmail.com.
References
- 1.Yeo-Teh N.S.L., Tang B.L. An alarming retraction rate for scientific publications on Coronavirus Disease 2019 (COVID-19) Account Res. 2020:1–7. doi: 10.1080/08989621.2020.1782203. [DOI] [PubMed] [Google Scholar]
- 2.Kambhampati S.B.S., Vaishya R., Vaish A. Unprecedented surge in publications related to COVID-19 in the first three months of pandemic: a bibliometric analytic report. J Clin Orthop Trauma. 2020;11:S304–S306. doi: 10.1016/j.jcot.2020.04.030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Gazendam A., Ekhtiari S., Wong E. The “infodemic” of journal publication associated with the novel coronavirus disease. The journal of bone and joint surgery. Am Vol. 2020;102:e64. doi: 10.2106/JBJS.20.00610. [DOI] [PubMed] [Google Scholar]
- 4.Emara K., Emara A.K., Farhan M., Mahmoud S. What orthopedic surgeons need to know about Covid-19 pandemic. J Orthop. 2020;21:275–277. doi: 10.1016/j.jor.2020.05.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Haneef R., Lazarus C., Ravaud P., Yavchitz A., Boutron I. Interpretation of results of studies evaluating an intervention highlighted in google health news: a cross-sectional study of news. PLoS One. 2015;10 doi: 10.1371/journal.pone.0140889. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Mehra M.R., Desai S.S., Ruschitzka F., Patel A.N. RETRACTED: hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. Lancet (London, England) 2020;395(10240):1820. doi: 10.1016/S0140-6736(20)31324-6. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(20)31324-6/fulltext [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Keshet D., Bernstein M., Dahan-Oliel N. Management of common elective paediatric orthopaedic conditions during the COVID-19 pandemic: the Montreal experience. J Child Orthop. 2020;14:161–166. doi: 10.1302/1863-2548.14.200107. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Wang Y.L., Zhu F.Z., Zeng L. Guideline for diagnosis and treatment of spine trauma in the epidemic of COVID-19. Chinese journal of traumatology = Zhonghua chuang shang za zhi. 2020;23:196–201. doi: 10.1016/j.cjtee.2020.06.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Khanduja V., Scarlat M.M. Reaching a new ’normal’ after COVID pandemic and orthopaedic implications. Int Orthop. 2020;44:1449–1451. doi: 10.1007/s00264-020-04725-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Abdelnasser M.K., Morsy M., Osman A.E. COVID-19. An update for orthopedic surgeons. Sicot-j. 2020;6:24. doi: 10.1051/sicotj/2020022. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Brand R.A. Reviewing for clinical orthopaedics and related research. Clin Orthop Relat Res. 2012;470:2622–2625. doi: 10.1007/s11999-012-2447-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Mavrogenis A.F., Quaile A., Scarlat M.M. The good, the bad and the rude peer-review. Int Orthop. 2020;44:413–415. doi: 10.1007/s00264-020-04504-1. [DOI] [PubMed] [Google Scholar]
- 13.Sprowson A.P., Rankin K.S., McNamara I., Costa M.L., Rangan A. Improving the peer review process in orthopaedic journals. Bone & Joint Res. 2013;2:245–247. doi: 10.1302/2046-3758.211.2000224. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Zazgyva A., Kon E., Mauffrey C., Mavrogenis A.F., Scarlat M.M. Reviews, reviewers and reviewing. Int Orthop. 2017;41:1–2. doi: 10.1007/s00264-016-3360-7. [DOI] [PubMed] [Google Scholar]
- 15.Mauffrey C., Scarlat M.M., Pećina M. Setting standards for medical writing in orthopaedics. Int Orthop. 2014;38:1–5. doi: 10.1007/s00264-013-2189-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
All relevant data are included in the manuscript.