The acquisition of Publons by Clarivate Analytics: questions at the heart of peer review
When Clarivate Analytics acquired Publons on 1 June 2017, there was much interest in understanding if this would result in either an evolution and improvement, or commodification, of peer review.1,2 The reason is because peer review tends to be exploitative, frequently extracting professional services, especially of peers and editors, for little or no monetary compensation while reaping record multi-billion dollar profits,3,4 so it is expected that the services, tools,5 and industries within and around academic journals and publishers also assume an exploitative nature. To compound this exploitative state, academia and academic publishing are currently experiencing a state of ‘fake’, including the abuse and fraud of peer review.6,7 It was hoped that a tool developed by Clarivate Analytics would be successful in combatting fraud in peer review.c However, the tool has failed to do so and its metrics arguably show biases and problems, as has been found for other research evaluation indicators.1
Concerned that the acquisition of Publons by Clarivate Analytics would further drive academia and scholarly publishing deeper into a more exploitative compensation scheme8 and unable to find answers to many questions regarding the inefficiencies of peer review, we decided to approach both Publons and Clarivate Analytics with a series of frank questions about issues that we believe need to be resolved: concerns that we had regarding the problem of fake or fraudulent peer reviews and the verification of peer reviewers’ expertise (Appendix 1, Appendix 2).
How has peer review evolved since the acquisition?
Publons' former mission statementd was clear (Figure 1), namely, to use experts to speed up research. However, the term ‘expert’ was removed from the mission statement in 2018 (Figure 1B and C),e which suggests that the quality/standards bar has been dropped rather than raised as anyone, including non-experts, can be valid peer reviewers at Publons, indicated by the fact that three of the biomedical journalsf who offer reviewer recognition with Publons accepted a dog as an editor in a sting operation.g They were EC Pulmonary and Respiratory Medicine,h Journal of Community Medicine & Public Health Care,i and Journal of Tobacco Stimulated Diseases.j On the other hand, the use of stings is unscholarly because it involves intrinsic false and misleading elements by the individuals conducting the operation.9
Figure 1.
An evolution in the mission statement of Publons suggests that standards are different, rather than improved, possibly to increase volumes. Whereas before July of 2017, peer reviewers were, according to the mission statement, expected to be experts (A), they were demoted (10 July) (B) to peer reviewers without necessarily having to be experts (C). Sources (screenshots of A, B, and C taken on 18 January 2018; D taken on 1 May 2019, and E on 20 December 2019 show different missions): (A) https://web.archive.org/web/20170702005520/https://publons.com/about/mission/; (B) https://web.archive.org/web/20170710025352/https://publons.com/about/mission; (C, D, E) https://publons.com/about/mission/.
In an editorial, Nassi-Calò10 remarked that the acquisition would enable editors to spot fake peer reviewers ‘by associating citation data from authors with their reviewing records in Publons’. Associating citation data with reviewing activity, we argue, could be used to develop a new metric, or another perverse incentive11 that would add pressure to already exploited and pressured academics. Our argument is reasonable if one examines more closely a previous viewpoint by Smith12 and another more recently by Wilkinson and Down.13 Smith stated: ‘Publons does not reward its users for the number and impact of published papers; instead, it ranks users [and institutions] on the basis of the number of papers they have peer reviewed’,12 while Wilkinson and Down13 suggested that universities can capture the quantity of peer review their researchers are already conducting. However, are there not risks in rewarding the quantity of peer review over its quality?8 Smith further noted that Publons' service will encourage scientists to contribute to the peer review process because many scientists like him could be ‘embarrassed to broadcast to everyone that [they] have published more papers than [they] have peer reviewed’. Arguments by Nassi-Calò and Smith that some potentially ‘predatory’ journals offer peer review recognition with Publons, as demonstrated above, raise important questions that we have sought to answer. If Publons can depend on editors to spot fictitious peer reviewers, then it might be able to weed out predatory peers,14 but if Publons rewards the quantity, rather than the quality of peer review, how will Publons and Clarivate Analytics improve it? How will journals and their editors, who failed to spot a fake editor,15 be able to effectively detect fake peer review? As co-winners of the 2017 ALPSP Award,k one would expect Publons to engage with academics and the public to provide concrete answers to such concerns. These concerns became more tangible, as was highlighted by a study that identified over 6,000, from a total sample of 183,743, ‘predatory’ reviews (i.e., reviews of Cabell-blacklisted ‘predatory’ journals) on Publons.16
There have been some notable achievements by both parties. Just prior to the acquisition, Clarivate Analytics partnered with ImpactStory (Figure 2),l which created oaDOI and Unpaywall.m On 5 October 2017, Publons partnered with The American Society for Microbiology.n In 2018, Publons launched an AI-powered tool ‘Publons Reviewer Connect’, which was described as a powerful tool to revolutionise editorial workflows for publisherso that combined ‘Publons' exclusive peer review database with the […] Web of Science author and citation index’. Reviewer Connect was developed to enable all publishers and journals, regardless of which peer review submission system they use, to ‘find, screen, and connect with expert peer reviewers’.p On 2 May 2019, Publons partnered with Clarivate Analytics' Web of Science ResearcherID.q However, unlike open peer review, traditional peer review reports are blind, not open access, so their quality cannot be verified.17,18
Figure 2.
On 23 June 2017, Clarivate Analytics partnered with ImpactStory claiming that ‘Novel public/private partnership connects researchers to verified versions of an estimated 18 million new open access articles from Web of Science’(A). The website where that announcement was made has been scrubbed clean, raising concerns about the erasure of documents that were publicly available, without any suitable explanation (B). The erasure of that press release further calls into question whether the partnership is still valid and active. Sources (screenshots taken on 24 June 2017 (A), 23 January 2018 (B)): http://news.clarivate.com/2017-06-23-Clarivate-Analytics-announces-landmark-partnership-with-Impactstory-to-make-open-access-content-easier-for-researchers-to-use (A, B).
Suggested changes to Publons to improve peer review transparency
There are several clearly positive aspects of Publons for authors who serve as reviewers, the most evident being a visible, public ‘reward’, in the form of recognition, for their effort as peer reviewers. Nonetheless, we recommend that Publons negotiates with publishers to ensure that all peer review reports are converted into open reports as this could validate the peer review activity of authors.19 Publons could also expose those who falsely claim to be peer reviewers of prestigious journals, so credits could be annulled for that author. Although Publons' COVID-19 indexr might assist experts to make sense of the flood of COVID-19 research, will critical reviews be encouraged by open review comments, and will negative scores for papers and preprints be permitted by Publons?
Conclusions
To detect fraud in peer review, Publons and Clarivate Analytics could use open peer review, digital identity verification, block fake reviewers’ accounts, blacklist fraudulent entities, and adopt fraud detection tools.s,t In essence, academics would like to know whether negative reviews, predatory reviewers, or failed peer reviews are rewarded at Publons.20
Source of funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors have no conflicts of interest to declare.
Ethical approval
There are no ethical or financial issues, conflicts of interest, or animal experiments related to this research.
Authors’ contributions
JATdS and AA equally conceived and designed the study, conducted background research, collected, organised, analysed, and interpreted information, and co-wrote all drafts of the paper. The authors have critically reviewed and approved the final draft of the article and are responsible for the manuscript's content and similarity index.
All website links were last verified on 17 January 2021.
Footnotes
Peer review under responsibility of Taibah University.
Publons' mission statement from 10 July 2017 to 18 January 2018: ‘Our mission at Publons is to speed up research by harnessing the power of peer review.’ https://publons.com/about/mission/. Note the subtle change (removal of the word “expert” from the mission statement) to ‘Our mission at Publons is to speed up research by harnessing the power of peer review’. https://web.archive.org/web/20171107164743/https://publons.com/about/mission/.
Curiously, the Publons mission page (20 December 2019) evolved to describe how Publons has partnered with academic publishers to help them give their peer reviewers the recognition they deserve (Figure 1E).
https://clarivate.com/news/clarivate-analytics-announces-landmark-partnership-impactstory-make-open-access-content-easier-researchers-use/ (a screenshot is provided in Figure 2).
https://www.asm.org/index.php/newsroom/item/6857-asm-and-publons-team-up-to-bring-recognition-to-peer-reviewers (URL now giving the following message ‘Not all of us live forever, including our pages, i.e., reference rot). The Publons site indicates that ASM is still an official partner: https://publons.com/publisher/42/american-society-for-microbiology.
https://publons.com/publon/covid-19/?sort_by=date (announced by email on 16 April 2020).
Appendix 1. Questions posed to Publons on 15 October 2017. Despite a reminder on 15 January 2018, no response has ever been received (content was edited only to remove any identifying names; no other content was edited).
-
1.
Can Publons confirm there are no fake peer review reports in the Publons database?
-
2.
Will the deal between Publons and Clarivate Analytics (i.e., the purchase of Publons by Clarivate Analytics on 1 June 2017) eliminate or reduce the problem of fake peer review and peer review rings? This is because fake peer review is sometimes provided by real peer reviewers who happen to be authors' friends. How does Publons propose to reduce or eliminate this problem?
-
3.
How is fake peer review defined by Publons?
-
4.
The problem with fake peer reviews is that they can include non-expert peer reviewers, or peer reviewers without subject matter expertise. How does Publons verify the expertise of peer reviewers, or does it allow any academic who has claimed to have ever conducted peer review to archive their peer review activity, irrespective of their expertise and/or academic/editorial experience, and gain credit for it?
-
5.
Has Publon's vision changed after the Clarivate Analytics acquisition? If yes, how?
-
6.
In your interview with Retraction Watch, you stated that editors will have access to peer reviewers' email addresses and will be able to view their reports. How then does Publons define confidentiality and privacy? Will editors pay to access and view that confidential information? Can reviewers make their own reports open access, despite signing confidentiality agreements with publishers, including some who are Publons sponsors? http://retractionwatch.com/2017/06/23/can-tracking-system-peer-reviewers-help-stop-fakes/
-
7.
Will the now-free practical peer review training course remain indefinitely free? https://publons.com/blog/publons-academy/
-
8.
Will graduates of Publons Academy have any obligation to provide a certain number of reviews?
-
9.
Who has access to peer review reports?
-
10.
Does sponsorship from some mainstream for-profit publishers not constitute financial and academic conflicts of interest?
Appendix 2. Questions posed to Clarivate Analytics on 15 October 2017. Despite a reminder on 15 January 2018, no response has ever been received.
-
1.
Will the deal between Publons and CA (i.e., the purchase of Publons by CA on 1 June 2017), eliminate or reduce the problem of fake peer review and peer review rings? This is because fake peer review is sometimes provided by real peer reviewers who happen to be authors' friends.
-
2.
Can CA elaborate and provide concrete examples about its previous statement, ‘Problems such as fraudulent scientific research and inefficiencies in peer review are among those that can be addressed using the combined strength of the two companies’: https://www.thebookseller.com/news/clarivate-analytics-buys-publons-562691
-
3.
Would CA consider using peer review reports to develop a future metric? If yes, how will the quality of peer review be measured, and how will this incorporate, or be associated with, the journal impact factor (JIF)?
-
4.
Who can access the list of journals with a JIF? Can authors, journalists, or the public access this list? If not, why not?
-
5.
Can CA enlighten researchers about the future of peer review, in particular the timeliness of peer review. Is this issue considered an inefficiency that CA would like to address, and if so, how does CA propose to achieve this?
References
- 1.Ortega J.L. Exploratory analysis of Publons metrics and their relationship with bibliometric and altmetric impact. Aslib J Inf Manage. 2019;71(1):124–136. [Google Scholar]
- 2.Teixeira da Silva J.A., Al-Khatib A. The ClarivateTM Analytics acquisition of Publons – an evolution or commodification of peer review? Res Ethics. 2019;15(3–4):438–444. [Google Scholar]
- 3.Johnson R., Watkinson A., Mabe M. 5th ed. The International Association of Scientific, Technical and Medical Publishers; The Hague, The Netherlands: 2018. The STM Report: an overview of scientific and scholarly publishing, October; p. 213.https://www.stm-assoc.org/2018_10_04_STM_Report_2018.pdf [Google Scholar]
- 4.Teixeira da Silva J.A., Dobránszki J., Tsigaris P., Al-Khatib A. Predatory and exploitative behaviour in academic publishing: an assessment. J Acad Libr. 2019;45(6):102071. [Google Scholar]
- 5.Martínez-López J.I., Barrón-González S., Martínez López A. Which are the tools available for scholars? A review of assisting software for authors during peer reviewing process. Publications. 2019;7(3):59. [Google Scholar]
- 6.Teixeira da Silva J.A. Fake peer reviews, fake identities, fake accounts, fake data: beware! AME Med J. 2017;2:28. [Google Scholar]
- 7.Trapp J. Predatory publishing, hijacking of legitimate journals and impersonation of researchers via special issue announcements: a warning for editors and authors about a new scam. Australasian Phys Eng Sci Med. 2020;43(1):9–10. doi: 10.1007/s13246-019-00835-5. [DOI] [PubMed] [Google Scholar]
- 8.Al-Khatib A., Teixeira da Silva J.A. Rewarding the quantity of peer review could harm biomedical research. Biochem Med. 2019;29(2) doi: 10.11613/BM.2019.020201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Al-Khatib A., Teixeira da Silva J.A. Stings, hoaxes and irony breach the trust inherent in scientific publishing. Publish Res Q. 2016;32(3):208–219. [Google Scholar]
- 10.Nassi-Calò L. In time: publons seeks to attract reviewers and improve peer review. Rev Paulista Pediat. 2017;35(4):367–368. [Google Scholar]
- 11.Edwards M.A., Roy S. Academic research in the 21st century: maintaining scientific integrity in a climate of perverse incentives and hypercompetition. Environ Eng Sci. 2017;34(1):51–61. doi: 10.1089/ees.2016.0223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Smith D.R. Will Publons popularize the scientific peer-review process? Bioscience. 2016;66(4):265–266. [Google Scholar]
- 13.Wilkinson J., Down P. Publons: Releasing the untapped power of peer review for universities. Insights. 2018;31:20. [Google Scholar]
- 14.Al-Khatib A., Teixeira da Silva J.A. Is biomedical research protected from predatory reviewers? Sci Eng Ethics. 2019;25(1):293–321. doi: 10.1007/s11948-017-9964-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Sorokowski P., Kulczycki E., Sorokowska A., Pisanski K. Predatory journals recruit fake editor. Nature. 2017;543(7646):481–483. doi: 10.1038/543481a. [DOI] [PubMed] [Google Scholar]
- 16.Severin A., Strinzel M., Egger M., Domingo M., Barros T. Who reviews for predatory journals? A study on reviewer characteristics. 2020. preprint, not peer reviewed. [DOI] [PMC free article] [PubMed]
- 17.Citrome L. Peer review and Publons – enhancements for the reviewer. Int J Clin Pract. 2016;70(5) doi: 10.1111/ijcp.12827. 364–364. [DOI] [PubMed] [Google Scholar]
- 18.Wicherts J.M. Peer review quality and transparency of the peer-review process in open access and subscription journals. PloS One. 2016;11(1) doi: 10.1371/journal.pone.0147913. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Teixeira da Silva J.A. Challenges to open peer review. Online Inf Rev. 2019;43(2):197–200. [Google Scholar]
- 20.Teixeira da Silva J.A. Are negative reviews, predatory reviewers or failed peer review rewarded at Publons? Int Orthop. 2020;44(10):2193–2194. doi: 10.1007/s00264-020-04587-w. [DOI] [PubMed] [Google Scholar]