Skip to main content
F1000Research logoLink to F1000Research
. 2024 Jul 9;13:439. Originally published 2024 May 2. [Version 2] doi: 10.12688/f1000research.148985.2

Paying reviewers and regulating the number of papers may help fix the peer-review process

Mohamed L Seghier 1,2,a
PMCID: PMC11221348  PMID: 38962691

Version Changes

Revised. Amendments from Version 1

Version 1 has been fully revised to address both reviewers’ comments. This has resulted in the addition of many new statements and references to Version 2. Specifically, the following changes have been made to: (1) clarify the implications of remunerating reviewers on the cost of scholarly communication as well as the (likely) risk of reviewers becoming uninterested in reviewing for not-for-profit publishers that may not afford to pay (or may pay less) compared to for-profit publishers, (2) expand the section about the need for publishers to reduce their APC if submission fees are levied, (3) add more details about the “Peer Community in” (PCI) initiative in Table 1, (4) edit the title and the Abstract to address reviewers’ comments, (5) acknowledge that peer reviewers are not all tenured academics and thus paying reviewers would help recognize the contribution from non-tenured staff, thereby making the peer review process more inclusive, (6) clarify that the exact amount for remunerating reviewers depend on many factors and that other bodies and agencies can play a role in regulating the payment of reviewers, (7) add a few statements about the expected roles of funding agencies if paying reviewers become the norm, and (8) proofread the main text to improve flow and readability.

Abstract

The exponential increase in the number of submissions, further accelerated by generative AI, and the decline in the availability of experts are burdening the peer review process. This has led to high unethical desk rejection rates, a growing appeal for the publication of unreviewed preprints, and a worrying proliferation of predatory journals. The idea of monetarily compensating peer reviewers has been around for many years; maybe, it is time to take it seriously as one way to save the peer review process. Here, I argue that paying reviewers, when done in a fair and transparent way, is a viable solution. Like the case of professional language editors, part-time or full-time professional reviewers, managed by universities or for-profit companies, can be an integral part of modern peer review. Being a professional reviewer could be financially attractive to retired senior researchers and to researchers who enjoy evaluating papers but are not motivated to do so for free. Moreover, not all produced research needs to go through peer review, and thus persuading researchers to limit submissions to their most novel and useful research could also help bring submission volumes to manageable levels. Overall, this paper reckons that the problem is not the peer review process per se but rather its function within an academic ecosystem dominated by an unhealthy culture of ‘publish or perish’. Instead of reforming the peer review process, academia has to look for better science dissemination schemes that promote collaboration over competition, engagement over judgement, and research quality and sustainability over quantity.

Keywords: peer review, research disseminations, referees, publishers, scholarly communication, awards and incentives, professional reviewers

Introduction

The peer review process has been the cornerstone of science dissemination for centuries. 1 , 2 It exists in different forms, 3 , 4 all based on the elusive concepts of referees’ impartiality 5 and competency. 6 The peer review process is known to be slow, expensive, inconsistent, biased, 3 , 7 11 and with limited impact on the quality of research publications. 12 , 13 These limitations have led to some calls to completely abandon the whole process because “ prepublication peer review is an enormous sink of scientists’ time, effort, and resources”. 14 Despite the limitations, one can note that the research community has still strong faith in it 7 , 15 ; i.e. “ despite the limitations of peer review process, we need it. It is all we have, and it is hard to imagine how we could get along without it”. 16 How to reform the peer review process has always been an extremely complex endeavor. 17 , 18

Many calls have been made to reform the peer review process, typically along four schools of thought according to Waltman (2023). Here, this review subscribes to the Efficiency & Incentives school that focuses on streamlining the peer review process and incentivizing participation in it. 19 This is a completely pragmatic choice because without an efficient peer review system, there is no point in my opinion in discussing its quality, reproducibility, transparency, or inclusion. Those qualities, promoted by other schools of thought, are extremely important but they are contingent on the suitability of the existing system to cope with the current research productivity levels of millions of scientific articles. 20 Specifically, over 2.5 million papers are published each year in English alone, 21 creating an incredible burden on editors and reviewers. The problem is obvious to all stakeholders: the exponential increase in the number of manuscripts 22 , 23 largely surpasses the current availability of qualified referees. Put another way, the current peer review process is not well equipped to serve modern publication dynamics that are expected to accelerate even more in this era of generative AI.

Finding adequate and sustainable alternative solutions is needed. As nature abhors a vacuum, if the current peer review model is not reformed, frustrated authors will have to rely on other means to get their studies out, in particular when desk rejection is reaching high rates. 24 , 25 This might for instance lead to a surge in alternative models based on the publication of unreviewed preprints, or the creation of local journals or repositories owned and managed by the authors’ universities and research institutions. 26 However, the biggest risk is the undesirable proliferation of predatory journals offering a “pay-to-publish” model sometimes without peer review. 27 29 I believe that models based on pre-publication peer-review are still useful as they do serve the interest of science relatively better than other models. 18 In that context, I discuss here the idea of promoting the role of professional reviewers, like professional editors, who will be monetarily compensated for their role in the peer-review process.

No more motivation to review for free

Below, I describe a concrete situation based on my own experience as editor-in-chief of a scientific journal. The journal I am editing, as might be the case for other journals, is experiencing a (welcome) growth in the number of submissions that are putting too much pressure on an already overstretched peer review process. 30 Consequently, this situation has resulted in (i) an unethical increase in desk rejection rates, (ii) a low percentage of invited researchers willing to review papers, (iii) an increase in the number of papers rejected after being sent for peer-review due to the difficulty to secure two or more referee reports, (iv) referees not submitting their reports after initially agreeing to do so, (v) referees submitting scant reports, listing superficial, unhelpful, or even irrelevant comments and suggestions, and (vi) an increase in instances of fraud where for example false contact information of potential referees with non-institutional emails was suggested or when the peer review process was manipulated. All these issues are directly or indirectly related to the pressure to ensure a fair and rigorous peer review process when the main players (reviewers) are no longer motivated to give away their time and expertise for free.

In this context, a wide range of solutions and innovations were proposed in the past (see analytical review in 31 ) to address the non-availability of volunteer peer reviewers. 31 35 , 18 , 36 , 37 Table 1 provides a succinct summary of some proposed innovations in the current literature. This includes for instance a more efficient involvement of editors in the selection of prospective reviewers, 38 improving diversity, 39 training reviewers and opening up the peer review process. 7 Others have argued that peer reviewers are rejecting invitations not necessarily because of lack of time but presumably because of lack of motivation. 40 Therefore, rewarding reviewers can help increase their (extrinsic) motivation, like offering certificates, discounts on publisher’s products, and publicly acknowledging the best and the most productive reviewers. 40 , 41

Table 1. Previously suggested solutions for improving the peer-review process.

A list of suggested solutions and innovations in previous studies. Paying professional reviewers can be implemented in conjunction with any of the solutions below.

Proposed solution Rationale and description
Implement peer review as an open, continuous, interactive forum involving all stakeholders This solution is based on two assumptions: (1) blinding reviewers to the identity of authors does not improve the quality of peer review, 80 and (2) peer review should promote engagement, not judgement. 81 Practically, the idea is to transform the peer review into an open and interactive scientific discussion 3 , 7 that operates in real time where concerned parties (authors, reviewers, editors) can interact via an open platform. This could involve diverse stakeholders 8 through multiple platforms such as blogs and social media. 1 This could be seen as one model of the open peer review framework 82 ). Ultimately, this model would promote constructive discussion to enable transparent editorial decisions that aggregate diverse opinions. 83
Put in place a hybrid two-tier system where only impactful research is sent for peer review This solution posits that not all submissions are ‘worth’ going through the prepublication peer review process. Instead, a hybrid model with a two-tier system is applied 79 : all submissions can be published and subjected to a post-publication peer review, but the prepublication peer review process needs only to evaluate manuscripts that are expected to have a significant impact on the field. 79 This is similar to some extent to desk-rejection practices where only submissions with high novelty and significance are sent for review. The main concern is of course how to make the initial evaluation about impact as objective as possible. This model, without safeguards, can end up biased toward top labs and renowned researchers. Moreover, it is not clear how citations from the unreviewed papers should be considered in the different research metrics.
Offer a non-selective review model to speed up the peer review process Reviewers are often asked to make judgments about other aspects of the paper beyond its scientific merit, including for instance rating its novelty, significance, impact, originality, readability, and even whether or not the paper will be highly cited if accepted. The non-selective review model mainly focuses on papers’ scientific quality rather than their perceived importance and novelty. 8 Therefore, peer reviewers only have to evaluate the methodological soundness of papers. It is also possible to go further along this model by asking reviewers only to review the methods and results section. The other sections (including the authors’ interpretations of the results in the Discussion section) can be open for discussion after publication.
Involve independent peer review platforms An initial evaluation of quality of papers is performed by other players. It is not unusual to hear peer reviewers complaining about the fact that they are often asked to review papers of poor quality. They believe it is not their job to improve papers as this should be the task of the original authors. In this era of publishing as many and as fast as possible, some submissions poorly report the undertaken work, which might make the work of reviewers even harder. Here the idea is to involve other parties to help the authors improve their submissions before submitting to a journal. This has been tested with the creation of independent peer review platforms, 3 helping researchers get their papers in good shape (for example, see platforms Rubriq and Editage). The authors pay the independent reviewers for their contribution to improving their papers. This model might add extra costs to the preparation process of papers.
Adopt a portable peer review across journals and publishers This idea has been implemented by different initiatives (e.g. the Neuroscience Peer Review Consortium, the Review Commons). It is based on having a ‘portable’ peer review across publishers. 84 In this model, a paper goes through a journal-independent peer review, then the authors revise their paper accordingly and submit both their revised paper and the review reports to any journal that is taking part in the portable peer-review initiative. The revised submission can be moved among journals until a final editorial decision is made. 35 This model might help editors use reviewers time more efficiently but might delay the publication of papers (e.g. a paper might remain stuck in the process until it is selected by a suitable journal).
Create communities of researchers reviewing preprints This model, promoted by “Peer Community in” (PCI), aims to organize the peer review process outside traditional journals. PCI is a non-profit scientific organization that creates communities of researchers to review and recommend preprints. It is based on an open-access system, a bottom-up process, and an active engagement of the scientific community. 85 It has strong support from many research institutions and funding agencies. As in the case of many not-for-profit initiatives, scalability would be an issue without sufficient resources.
Merge journals into mega-journals with large editorial boards A mega-journal (like PLOS One and Nature’s Scientific Reports) has a very broad scope, accepting articles in almost all domains. It typically includes a large editorial board and a very large pool of reviewers. Having relatively low rejection rates, mega-journals usually publish thousands of articles per year. They tend to have an efficient (fast turnaround time) peer review process that can handle large submission volumes. 3 They sometimes play a similar role as a cascade journal, publishing papers rejected by more selective top journals. 86 Because they rely on a large number of academic editors across multiple domains, they can efficiently involve peer reviewers. However, this mega-journal model might not serve the interests of all domains, as specialized journals are needed.
Build an online reviewer registry accessible by all journals The idea here is to build a large online registry of volunteer reviewers 87 who can be invited by any journal that has access to the registry. This registry can be created in a collaborative way by different journals or publishers, including for instance consortia of journals with overlapping scope or domains. 87 Such registries can complement the reviewers databases that journals hold. The registry can include reviewers from diverse geographical locations, demographics and expertise. To sustain such initiatives, a cross-publisher partnership has to oversee the creation and maintenance of the registry, with the mission to support it with the right resources.
Augment the review process with technology Technology has already modernised the peer review process 88 ; e.g. automated detection of flawed methods or unethical practices like inappropriate statistical analysis, figure manipulation, or data fabrication. One can note the growing interest in the development of powerful and versatile software and platforms to streamline the peer-review process (e.g. Scholastica). It is expected that the adoption of generative AI will have many ramifications on the peer review process in the near future. 89 , 90
You review as much as you publish As the number of prolific (very productive) researchers is growing very fast, 77 this model requires active involvement from such frequent users of the system. Therefore, researchers who publish more should contribute more to the peer review process. 91 It is not unusual to see many senior authors who publish a lot (and hence who are putting too much pressure on the system) are not involved at all in the peer review process. Some have even suggested that senior authors must provide evidence of their contribution to the peer review process as a requirement for considering their manuscripts for review. 91 This idea, albeit attractive, might not be feasible as it ignores the fact that senior authors are also very often involved in other academic review duties (doctoral theses, grant applications).
Offer incentives and rewards to reviewers There is already a variety of rewards offered to reviewers as a recognition of their contribution to the peer review process. For instance, this includes prizes, discounts on publication fees, free access to the publisher’s products (e.g. databases), vouchers for book purchases, reduced registration fees for conferences, recognition in databases, and many others. Moreover, active reviewers who are submitting quality reports can be added to the editorial board of the concerned journals. Despite these many incentives, one cannot deny that the number of available reviewers is still declining.
Link peer review involvement to the researcher’s academic benefits Other incentives argue for linking promotion applications (e.g. for tenured positions) to how active the researcher is in the peer review process. 92 Other suggestions include the possibility of earning continuing education credits upon completion of a review or waiving publication fees in the journal. 93 Others have even suggested the idea of offering co-authorship to reviewers, 94 in particular when the reviewers’ suggestions had significantly improved the original submission. However, this model might exclude those reviewers who are not ‎tenured academics.
Abandon the prepublication peer review model and adopt the post-publication peer-review (PPPR) model The post-publication peer-review (PPPR) model 95 , 96 can ease the burden on journals with large submission volumes. It does not consider rejection after peer review. There is already rich literature about this model, 82 and it has been tested experimentally by different initiatives (e.g. F1000Research, ScienceOpen, The Winnower). Despite its many merits, this model has some limitations as it might let all papers be available in the public domain, regardless of their original quality and impact. Without safeguards and an initial thorough check of paper quality, this model might end up mimicking the work of predatory journals that publish all submissions without (prepublication) peer review. Hence, PPPR must allow for continuous discussion and improvement of the quality of publications. 95 However, there has to be a limit on when a paper can remain open for discussion as progress in science will lead to new experimental protocols and novel conceptual frameworks that will make ‘old’ publications look erroneous or obsolete.
Abandon the peer review process altogether The (prepublication) peer review process should be abandoned as there is no strong evidence about its usefulness (e.g. see discussion in Ref. 14). The alternative is to gradually move toward expanding the role of preprint servers 8 , 26 or allowing universities and research institutions to manage their researchers’ publications. It is the researchers who create knowledge and it is thus the responsibility of their institutions to disseminate their work in the format they deem fit.

However, these suggestions might not solve existing problems for four main reasons: (i) the number of submissions is projected to increase at much faster rates in this era of generative AI, (ii) the roles and duties of the academics include administration, writing grants, writing papers, writing patents, commercialization ventures, leaderships and directorships, editorships, teaching, supervision, committee memberships, consultancy, and outreach, leaving no time for reviewing papers (iii) the number of journals and publications in languages other than English is growing fast, promoted by current initiatives to improve inclusion in science dissemination, which makes it challenging to ensure a rigorous multilingual peer review process, (iv) researchers avoid peer-reviewing because they are increasingly unsatisfied ‎with for-profit publishers that reap exorbitant ‎profits while contributing negligibly towards academia and research, and (v) perhaps most importantly, researchers are likely to become less willing to spend time on reviewing AI-enhanced quick-to-write papers (e.g. when the time taken to review a paper becomes longer than the time taken to draft that paper with AI). I believe the latter is likely to be increasingly present in the debate about reforming the peer review process, which is why monetary incentives might be one potential means to alleviate the burden on the peer review process to some extent.

Promoting professional reviewing as a career

Publishing research articles is still an expensive business even in this digital era. 42 44 The global annual cost of peer-review is estimated at around $1.5 billion, 45 corresponding to a net annual contribution of over 100 million hours by reviewers. 46 In the current science dissemination business evaluated at billions of dollars, peer reviewers are still providing volunteer labour. Journals should thus develop methods to compensate reviewers for their time while maintaining the integrity of the peer-review process. 45 , 47 The system needs fair and sustainable mechanisms to pay referees. This would allow researchers to earn money by becoming full-time or part-time professional reviewers. Like the remunerated role played by professional language editors, researchers who enjoy evaluating research papers can be invited to dedicate more time to the peer review process for a fee. 48 For instance, a professional reviewer can be tasked with reviewing a given number of papers per week (e.g. 2 or 3 papers/week), which would help speed up the peer review process in particular for journals with large submission volumes.

We already have models where referees are paid for reviewing academic work, as grant reviewers, PhD examiners, or academic promotion examiners. Reviewing articles is no exception and could thus be considered a remunerated task in academia. There is room for the emergence of two different but complementary professional reviewers. Reviewers can still be academicians, i.e. tenured staff and academics, working in universities and research institutions, where they can be offered the opportunity to supplement their salaries with additional honorariums for their active involvement in the peer review process. Another alternative concerns the possibility of outsourcing the peer review process by allowing the creation of independent providers. These providers, managed by for-profit companies, 49 can follow the same work model as companies that employ professional language editors in research dissemination. The job of a professional reviewer can be regulated by adopting new ethical and professional standards to safeguard the peer review process from any possible unethical practices that might occur when money is involved in the process ( Figure 1). Furthermore, peer-reviewing is also undertaken by non-tenured staff and academics with relevant ‎qualifications and expertise (i.e. postdocs, senior technical staff, research assistants, laboratory engineers and managers, ‎ and PhD-qualified persons who have quit academia). However, they generally cannot claim academic promotions for their voluntary participation in ‎the peer-review process. Thus, an inclusive peer review process must acknowledge their contributions by financially rewarding ‎them.

Figure 1. Professional reviewers can be an integral part of the peer review process.

Figure 1.

A typical publication process that involves professional reviewers. It shows the different steps (blue boxes) and their implementation in a fair and transparent way (green boxes).

The proponent of this idea argues that peer review should be considered as a business transaction, 50 where a modest remuneration per paper of around 200$ 50 , or $450 for for-profit publishers, 51 can be offered to reviewers. This can motivate reviewers to be more actively involved in the peer review process and to offer fast and thorough review reports. 52 , 53 This can also be an incentive for retired scientists 50 , 54 to participate in order to gain an extra income that expands their retirement plans. Paying referees can increase the pool of available reviewers, including for instance researchers who cannot afford to work for free. 54 Many factors may interplay to determine the level of remuneration, 46 , 55 , 56 including reviewing for not-for-profit versus for-profit journals, scientific domains (e.g. research in medicine is more funded and expensive than research in art and humanities), the number of reviewers required per manuscript (e.g. two or more referees), the type of articles (e.g. reviewing a research article requires more hours than a commentary), and their length and complexity (e.g. reviewing a long review article or a full research article with complex data is more demanding than an opinion piece). Publishers and editors should estimate the level of remuneration according to the average hours typically spent by referees for each article type 46 and the average yearly number of manuscripts reviewed by each reviewer. 37 , 56 Other organizations and agencies can also establish general guidelines about the payment of reviewers, including the Committee on Publication Ethics, the Declaration on Research Assessment, the Confederation of Open Access Repositories, and the European Association of Science Editors. Below, I discuss some options to sustain the payment of professional referees.

Reviewers as consultants: Researchers are usually invited to budget for the support of a consultant or a technical expert when applying for grants. For instance, the consultant can provide expert opinion about an aspect of the research project that might be beyond the expertise of the researchers (e.g. a legal advisor, an industrial partner, a clinical professional, a computer system expert, a programmer, … etc). In the same way, a reviewer can be seen as a consultant whose role is to support the researchers in improving their papers in terms of readability, quality, and methodological soundness. Publications are typically included as deliverables in research grants, and therefore the contribution of peer reviewers to improve these deliverables can be budgeted as consultancy fees. Funding bodies may categorize this type of expense as publication fees. However, regardless of how paying reviewers is included in the budget, too much financial burden will be put on funding bodies as they are already covering the excessive article-processing charges. Similar to some alternatives being sought for article-processing charges, 57 funding bodies should put in place guidelines to make the payment process of professional reviewers as fair and transparent as possible, in particular when for-profit agencies are managing the peer review process. For instance, many funding bodies already require that published work must be made freely available, and thus similar requirements can also be made about the type of peer review (free or paid). If there is strong evidence that paying reviewers improves the quality of publications, then funding bodies can recommend researchers to publish in journals that pay referees. This ensures that funded research is being evaluated and scrutinized by the most rigorous review process that scholarly communication can offer (and afford).

Levying a submission fee: As is the case in some journals, a submission fee can be collected for each submitted article, which can then be used to pay professional reviewers. For example, around 45 of the economics, finance and accounting journals published by Elsevier charge a submission fee of around $50-$100. Likewise, some biology journals also levy submission fees. Their rationale is to ensure that only within-scope and high-quality papers are received. Submission fees also serve other purposes like sponsoring conferences and workshops, the payment of language editing services, prizes/awards such as the annual best paper award, and even payments to top reviewers. To make payment of professional reviewers sustainable, it might be suggested to levy a submission fee of $200 per submitted paper. This fee should only be charged for submissions sent for review. I believe a $200 would be a fair submission fee for the service of getting constructive comments about how to improve a paper. For example, for a journal with a submission volume of 1,000 papers per year, and a desk-rejection rate of 50%, the collected submission fees (around $100,000) can help secure the services of professional reviewers. To make this option attractive for authors, the submission fee should be deduced from the article-processing charges or the open access fee once the peer-reviewed paper is accepted. Consequently, this might mean that publishers should consider reducing their article processing charges when levying a submission fee. However, this might sound unsavory for for-profit publishers and hence the community needs to create alternative models an ecosystem that would make publishers active contributors to peer review, as discussed below.

Paying referees in the context of economic disparity: One of the consequences of paying reviewers might be a better diversity in peer review. This is because paying referees might be very appealing for researchers from low- or middle-income countries. However, this monetary incentive can inadvertently yield over-involvement in the peer review process because a fee of 200$ or more is sometimes comparable to a full salary in some low-income countries. Put another way, researchers from low-income countries can favour paid peer review over other academic duties, which might not be a tenable situation for their institutions. One way to avoid this race for more reviews, and hence more money, is to link the number of invitations for referees from low-income countries to the size of authors or readers from those low-income countries. For instance, if a journal has a readership of 20% from low-income countries, the number of invited referees from low-income countries can be capped at twice this percentage. Furthermore, all paid review tasks should be declared on the publisher’s website so that editors are well informed about the level of involvement of different referees. This would help identify any unusual review dynamics that need adjustments to ensure a fair and robust peer review.

How about the publishers? The research community has voiced clear dissatisfaction about the ‘parasitic’ behaviour of some for-profit publishers in research dissemination. They are seen as the biggest winners of the current system despite not funding research and not paying the original content creators (the authors) and evaluators (the reviewers). 58 Recent examples of editors quitting for-profit journals provide an interesting example to appreciate the depth of that dissatisfaction (e.g. Refs. 59, 60). I believe that, if for-profit publishers do not engage with the research community more transparently and constructively, they will soon be made irrelevant to the whole process. Publishers must realize that they cannot continue operating along the same model. For instance, they can inject some of their profits back into the peer review process to support the remuneration of reviewers. However, it is unlikely that publishers would be willing to financially contribute to the peer review process unless they have control over the process through the creation of their professional review agencies. However, I don’t think the research community can again trust for-profit publishers with the peer review process, which is understandable given the previous experience of the community with the open access model that has unfortunately been made so expensive and unaffordable by some for-profit publishers.

In this context, I feel that scholarly communication might foreseeably follow a radical model by completely bypassing for-profit publishers, i.e. a model that does not need the ‘middleman’. This implies that researchers and their universities will own the whole process. 61 In this model, universities and research institutions might choose to manage the whole publication process, for instance in the form of posting papers on open repositories, 62 while adopting an open post-publication peer review model. Universities can work as consortia to share the cost of publication through common or shared repositories, which can be further facilitated by the adoption of generative AI in the editorial process. 63 , 64 The other implication here is to allow researchers (or their institutions) to keep the copyright of their work instead of conceding it to publishers as in the current model.

Arguments against paying peer reviewers: The opponents to paying reviewers mention the risks of proliferation of unethical practices (i.e. the system being corrupted by money) and unsustainable costs 51 , 54 , 65 as it can lead to increases in subscription fees and article processing charges. 66 Moreover, adding a monetary component to the peer review might lead to unethical practices to maximize the number of completed reviews such as non-disclosure of conflict of interest, the tendency to review papers even outside one’s domain of expertise, and a fast turnaround time with no thorough evaluation of the reviewed papers. Therefore, journals should put some safeguards to ensure a high quality and rigorous peer review process, for instance, through the evaluation/rating by the editors of completed reviews before approving payment to the referees.

Likewise, paying reviewers might yield a proliferation of for-profit peer review agencies with dubious ethical practices, like paper mills, that can generate fabricated review reports. 66 Some studies have argued that monetary rewards tend to decrease the quality of the peer review process 67 as well as the intrinsic motivation of researchers to contribute to the peer review process. 68 Paying reviewers might be only sustainable for journals with a reasonable submission volume. 69 For multidisciplinary journals with broad topics and large submission volumes, the peer review process can be enormously costly and thus diverse funding sources have to be sought. There is also the risk that paying referees will soon become the norm, making researchers no longer interested in reviewing for free. It could also lead to a multi-tier peer review system depending on how much a journal chooses to pay its reviewers, hence engendering an unequal competition among journals for the service of reviewers. This could hurt not-for-profit publishers that cannot afford an expensive peer review process. In that context, not-for-profit publishers must (i) diversify incentives on top of paying reviewers, (ii) nurture the intrinsic motivation of their readers to participate in peer review and serve their scientific community, 70 (iii) seek alternative means to be subsidized, 71 and (iv) foster strategic partnerships with all stakeholders to make not-for-profit journals the preferred model for open-access publicly-funded research. Overall, although one cannot rule out the risk of corruption in peer review by money, I trust that academia has built-in mechanisms to handle financial transactions fairly and objectively as it is the case when researchers are paid to evaluate grant proposals and academic theses. 54

Other schemes of financial compensation: It might be worth mentioning other alternative models to professional reviewers. For example, one proposal argues for offering monetary compensation not for the reviewers themselves but for their institutions 72 as they usually deal with the same publishers for purchasing subscriptions and other products. The rationale is that reviewers are already paid by their institutions and hence their involvement in the peer review process should benefit their institutions. However, this sometimes ignores the fact that researchers sometimes (often?) conduct their reviews outside work hours (e.g. evenings or weekends), and hence their contributions to peer review should not be assumed already covered by their salaries. Likewise, another recent proposal argues for monetary prizes not to individual referees but to their groups (departments, research units or labs). 68 That collected money for instance can support ongoing research projects, support early career members or the work of members from underrepresented minorities. 68

Peer review cannot thrive in the current academic ecosystem

The above-mentioned solutions are assumed remedies to a not-fit-for-purpose peer review process. But maybe the whole peer review process is fine, and all encountered problems are because peer review is operating within the wrong environment. Indeed, the culprit here is the current academic environment that promotes an unhealthy culture of “publish or perish”. To survive in this culture, researchers might fragment their results into multiple publications or even publish redundant papers based on similar data. 35 , 73 This culture of “publish or perish” is hurting academic research. 74 Therefore, instead of instigating calls to abandon or reform the peer review process, academia should invent and campaign for new models that depart from the “publish or perish” model. 75 As long as research quality is measured as a function of research productivity, there will be no form of peer review that can guarantee quality research while dealing with a deluge of papers that is now being inflated even more by the adoption of generative AI.

Is the ultimate goal of academia to produce prolific researchers who can publish several papers a week, 76 sometimes with questionable practices? 77 If, hypothetically, this goal is achieved by every researcher, who would review (and read) the published work? Evidence shows that many published papers remain uncited or end up having little impact on their respective fields, 78 but one cannot ignore the resources those papers took (time and effort) from reviewers and editors. It is as if the whole purpose of many substandard papers is to inflate some research metrics. Can academia promote a new model that puts a cap on how many papers an author can submit per year? This is essentially to invite researchers to think sensibly about the quality of their research and maybe only publish the best of their research that is likely to have an impact on their respective fields. Other work can still be shared as unreviewed preprints. 79 This framework ensures a manageable number of submissions, though it might entail a rethinking of current citation-based metrics of research impact and productivity.

Conclusion

We can’t afford to ignore existing flaws in the current peer review system because the status quo will only favour the emergence of unethical practices. Here, I support the idea, albeit not new, of the creation of professional peer reviewers. Paying reviewers must be made fairly and transparently. Future work needs to investigate alternative ways to ensure a sustainable and affordable remuneration of professional reviewers. The discussion should continue among all stakeholders to identify new solutions and innovations for efficient and affordable research dissemination in the modern era of generative AI. Empowering universities and research institutions to own the whole research dissemination system might soon be an inevitable scheme to consider. Finally, without reforming academia when it comes to disseminating the work of its members, scholarly communication will continue relying on a costly and ineffective peer review system that will sooner or later be completely abandoned or replaced by something else.

Contributions

This opinion article was conceived and written by MLS.

Funding Statement

This work was funded by Khalifa University of Science and Technology [Grant Number: RC2-2018-022]

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

[version 2; peer review: 2 approved

Data availability

No data are associated with this article.

References

  • 1. Castillo M: Peer review: past, present, and future. AJNR Am. J. Neuroradiol. 2012;33(10):1833–1835. 10.3174/ajnr.A3025 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Spier R: The history of the peer-review process. Trends Biotechnol. 2002;20(8):357–358. 10.1016/S0167-7799(02)01985-6 [DOI] [PubMed] [Google Scholar]
  • 3. Jubb M: Peer review: The current landscape and future trends. Learn. Publ. 2016;29(1):13–21. 10.1002/leap.1008 [DOI] [Google Scholar]
  • 4. Koshy K, Fowler AJ, Gundogan B, et al. : Peer review in scholarly publishing part A: why do it? Int. J. Surg. Oncol. 2018;3(2):e56. 10.1097/IJ9.0000000000000056 [DOI] [Google Scholar]
  • 5. Atkinson M: Regulation of Science by “Peer Review”. Stud. Hist. Phil. Sci. 1994;25:147–158. 10.1016/0039-3681(94)90025-6 [DOI] [Google Scholar]
  • 6. Baxt WG, Waeckerle JF, Berlin JA, et al. : Who reviews the reviewers? Feasibility of using a fictitious manuscript to evaluate peer reviewer performance. Ann. Emerg. Med. 1998;32(3 Pt 1):310–317. 10.1016/S0196-0644(98)70006-X [DOI] [PubMed] [Google Scholar]
  • 7. Smith R: Peer review: a flawed process at the heart of science and journals. J. R. Soc. Med. 2006;99(4):178–182. 10.1177/014107680609900414 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8. Walker R, Rocha da Silva P: Emerging trends in peer review-a survey. Front. Neurosci. 2015;9:169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9. Haffar S, Bazerbachi F, Murad MH: Peer Review Bias: A Critical Review. Mayo Clin. Proc. 2019;94(4):670–676. 10.1016/j.mayocp.2018.09.004 [DOI] [PubMed] [Google Scholar]
  • 10. Rooyen S: A critical examination of the peer review process. Learn. Publ. 1998;11(3):185–191. 10.1087/09531519850146355 [DOI] [Google Scholar]
  • 11. Williamson A: What will happen to peer review? Learn. Publ. 2003;16(1):15–20. 10.1087/095315103320995041 [DOI] [Google Scholar]
  • 12. Jefferson T, Wager E, Davidoff F: Measuring the quality of editorial peer review. JAMA. 2002;287(21):2786–2790. 10.1001/jama.287.21.2786 [DOI] [PubMed] [Google Scholar]
  • 13. Wager E, Jefferson T: Shortcomings of peer review in biomedical journals. Learn. Publ. 2001;14(4):257–263. 10.1087/095315101753141356 [DOI] [Google Scholar]
  • 14. Heesen R, Bright LK: Is Peer Review a Good Idea? Br. J. Philos. Sci. 2021;72(3):635–663. 10.1093/bjps/axz029 [DOI] [Google Scholar]
  • 15. Jefferson T, Alderson P, Wager E, et al. : Effects of editorial peer review: a systematic review. JAMA. 2002;287(21):2784–2786. 10.1001/jama.287.21.2784 [DOI] [PubMed] [Google Scholar]
  • 16. Relman AS: Peer review in scientific journals--what good is it? West. J. Med. 1990;153(5):520–522. [PMC free article] [PubMed] [Google Scholar]
  • 17. Chloros GD, Giannoudis VP, Giannoudis PV: Peer-reviewing in Surgical Journals: Revolutionize or Perish? Ann. Surg. 2022;275(1):e82–e90. 10.1097/SLA.0000000000004756 [DOI] [PubMed] [Google Scholar]
  • 18. Kelly J, Sadeghieh T, Adeli K: Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide. EJIFCC. 2014;25(3):227–243. [PMC free article] [PubMed] [Google Scholar]
  • 19. Waltman L, Kaltenbrunner W, Pinfield S, et al. : How to improve scientific peer review: Four schools of thought. Learn. Publ. 2023;36(3):334–347. 10.1002/leap.1544 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20. Jinha AE: Article 50 million: an estimate of the number of scholarly articles in existence. Learn. Publ. 2010;23(3):258–263. 10.1087/20100308 [DOI] [Google Scholar]
  • 21. Tennant JP: The state of the art in peer review. FEMS Microbiol. Lett. 2018;365(19). 10.1093/femsle/fny204 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22. Larsen PO, Ins M: The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics. 2010;84(3):575–603. 10.1007/s11192-010-0202-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23. Bornmann L, Haunschild R, Mutz R: Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanit. Soc. Sci. Commun. 2021;8:224. 10.1057/s41599-021-00903-w [DOI] [Google Scholar]
  • 24. Meyer HS, Durning SJ, Sklar DP, et al. : Making the First Cut: An Analysis of Academic Medicine Editors’ Reasons for Not Sending Manuscripts Out for External Peer Review. Acad. Med. 2018;93(3):464–470. 10.1097/ACM.0000000000001860 [DOI] [PubMed] [Google Scholar]
  • 25. Seghier ML: Demystifying desk rejection: A call to action for our authors. Int. J. Imaging Syst. Technol. 2022;32(3):701–703. 10.1002/ima.22733 [DOI] [Google Scholar]
  • 26. Gogotsi Y: Pay to publish? Open access publishing from the viewpoint of a scientist and editor. Graphene 2D Mater. 2023;8:1–3. 10.1007/s41127-023-00057-3 [DOI] [Google Scholar]
  • 27. Camargo LM, Smirnov M, Maldonado IL: The prey’s perspective on the rise of predatory publishing. EXCLI J. 2023;22:904–906. 10.17179/excli2023-6392 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28. Grudniewicz A, Moher D, Cobey KD, et al. : Predatory journals: no definition, no defence. Nature. 2019;576(7786):210–212. 10.1038/d41586-019-03759-y [DOI] [PubMed] [Google Scholar]
  • 29. Mertkan S, Onurkan G, Nilgun Suphi G: Profile of authors publishing in ‘predatory’ journals and causal factors behind their decision: A systematic review. Res. Eval. 2021;30(4):470–483. 10.1093/reseval/rvab032 [DOI] [Google Scholar]
  • 30. Seghier ML: Demystifying desk rejection: A call to action for our authors. Int. J. Imaging Syst. Technol. 2022;32(3):701–703. 10.1002/ima.22733 [DOI] [Google Scholar]
  • 31. Kaltenbrunner W, Pinfield S, Waltman L, et al. : Innovating peer review, reconfiguring scholarly communication: an analytical overview of ongoing peer review innovation activities. J. Doc. 2022;78(7):429–449. 10.1108/JD-01-2022-0022 [DOI] [Google Scholar]
  • 32. Woods HB, Brumberg J, Kaltenbrunner W, et al. : An overview of innovations in the external peer review of journal manuscripts. Wellcome Open Res. 2022;7:82. 10.12688/wellcomeopenres.17715.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33. Spier RE: Peer review and innovation. Sci. Eng. Ethics. 2002;8(1): p.99–108. discussion 109-12. 10.1007/s11948-002-0035-0 [DOI] [PubMed] [Google Scholar]
  • 34. Barroga E: Innovative Strategies for Peer Review. J. Korean Med. Sci. 2020;35(20):e138. 10.3346/jkms.2020.35.e138 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35. Stahel PF, Moore EE: Peer review for biomedical publications: we can improve the system. BMC Med. 2014;12:179. 10.1186/s12916-014-0179-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36. Nguyen VM, Haddaway NR, Gutowsky LF, et al. : How long is too long in contemporary peer review? Perspectives from authors publishing in conservation biology journals. PLoS One. 2015;10(8): e0132557. 10.1371/journal.pone.0132557 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37. Fernandez-Llimos F, Salgado TM, Tonin FS: How many manuscripts should I peer review per year? Pharm. Pract (Granada). 2020a;18(1):1804. 10.18549/PharmPract.2020.1.1804 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38. Zupanc GKH: “It is becoming increasingly difficult to find reviewers”—myths and facts about peer review. J. Comp. Physiol. A. 2023;210:1–5. in press. 10.1007/s00359-023-01642-w [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39. Ben Messaoud K, Schroter S, Richards M, et al. : Analysis of peer reviewers’ response to invitations by gender and geographical region: cohort study of manuscripts reviewed at 21 biomedical journals before and during covid-19 pandemic. BMJ. 2023;381:e075719. 10.1136/bmj-2023-075719 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40. Ellwanger JH, Chies JAB: We need to talk about peer-review-Experienced reviewers are not endangered species, but they need motivation. J. Clin. Epidemiol. 2020;125:201–205. 10.1016/j.jclinepi.2020.02.001 [DOI] [PubMed] [Google Scholar]
  • 41. Künzli N, Berger A, Czabanowska K, et al. : «I Do Not Have Time»—Is This the End of Peer Review in Public Health Sciences? Public Health Rev. 2022;43:1605407. 10.3389/phrs.2022.1605407 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42. Grossmann A, Brembs B: Current market rates for scholarly publishing services [version 2; peer review: 2 approved]. F1000Res. 2021;10:20. 10.12688/f1000research.27468.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43. Smart P: Peer review: An expensive business. Learn. Publ. 2016;29:3–4. 10.1002/leap.1012 [DOI] [Google Scholar]
  • 44. Donovan B: The truth about peer review. Learn. Publ. 1998;11(3):179–184. 10.1087/09531519850146346 [DOI] [Google Scholar]
  • 45. LeBlanc AG, Barnes JD, Saunders TJ, et al. : Scientific sinkhole: estimating the cost of peer review based on survey data with snowball sampling. Res. Integr. Peer Rev. 2023;8(1):3. 10.1186/s41073-023-00128-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46. Aczel B, Szaszi B, Holcombe AO: A billion-dollar donation: estimating the cost of researchers’ time spent on peer review. Res. Integr. Peer Rev. 2021;6(1):14. 10.1186/s41073-021-00118-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47. Gasparyan AY, Gerasimov AN, Voronov AA, et al. : Rewarding peer reviewers: maintaining the integrity of science communication. J. Korean Med. Sci. 2015;30(4):360–364. 10.3346/jkms.2015.30.4.360 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 48. Ott S, Hebenstreit D: Supply and demand: Apply market forces to peer review. Nature. 2014;506(7488):295. 10.1038/506295b [DOI] [PubMed] [Google Scholar]
  • 49. Van Noorden R: Company offers portable peer review. Nature. 2013;494(7436):161. 10.1038/494161a [DOI] [PubMed] [Google Scholar]
  • 50. Diamandis EP: Publishing costs: Peer review as a business transaction. Nature. 2015;517(7533):145. 10.1038/517145a [DOI] [PubMed] [Google Scholar]
  • 51. Brainard J: The $450 question: Should journals pay peer reviewers? Science. 2021. 10.1126/science.abh3175 [DOI] [Google Scholar]
  • 52. Hamermesh DS: Facts and Myths about Refereeing. J. Econ. Perspect. 1994;8(1):153–163. 10.1257/jep.8.1.153 [DOI] [Google Scholar]
  • 53. Thompson GD, Aradhyula SV, Frisvold G, et al. : Does Paying Referees Expedite Reviews?: Results of a Natural Experiment. South. Econ. J. 2010;76(3):678–692. 10.4284/sej.2010.76.3.678 [DOI] [Google Scholar]
  • 54. Cheah PY, Piasecki J: Should peer reviewers be paid to review academic papers? Lancet. 2022;399(10335):1601. 10.1016/S0140-6736(21)02804-X [DOI] [PubMed] [Google Scholar]
  • 55. Huisman J, Smits J: Duration and quality of the peer review process: the author’s perspective. Scientometrics. 2017;113:633–650. 10.1007/s11192-017-2310-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56. Yankauer A: Who are the peer reviewers and how much do they review? JAMA. 1990;263(10):1338–1340. 10.1001/jama.1990.03440100042005 [DOI] [PubMed] [Google Scholar]
  • 57. Sanderson K: Who should pay for open-access publishing? APC alternatives emerge. Nature. 2023;623(7987):472–473. 10.1038/d41586-023-03506-4 [DOI] [PubMed] [Google Scholar]
  • 58. Walter P, Mullins D: From symbiont to parasite: the evolution of for-profit science publishing. Mol. Biol. Cell. 2019;30(20):2537–2542. 10.1091/mbc.E19-03-0147 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 59. Sanderson K: Editors quit top neuroscience journal to protest against open-access charges. Nature. 2023;616(7958):641. 10.1038/d41586-023-01391-5 [DOI] [PubMed] [Google Scholar]
  • 60. Singh Chawla D: Open-access row prompts editorial board of Elsevier journal to resign. Nature. 2019. 10.1038/d41586-019-00135-8 [DOI] [Google Scholar]
  • 61. Roh C: Owning the peer review process. Coll. Res. Libr. News. 2022;83(3). 10.5860/crln.83.3.100 [DOI] [Google Scholar]
  • 62. Torres L, Hartley R: Repositories for academic products/outputs: Latin American and Chilean visions. F1000Res. 2019;8:1517. 10.12688/f1000research.19976.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63. Conroy G: How ChatGPT and other AI tools could disrupt scientific publishing. Nature. 2023;622(7982):234–236. 10.1038/d41586-023-03144-w [DOI] [PubMed] [Google Scholar]
  • 64. Kousha K, Thelwall M: Artificial intelligence to support publishing and peer review: A summary and review. Learned Publishing. 2024;37(1):4–12. 10.1002/leap.1570 [DOI] [Google Scholar]
  • 65. Gropp RE, Glisson S, Gallo S, et al. : Peer Review: A System under Stress. Bioscience. 2017;67(5):407–410. 10.1093/biosci/bix034 [DOI] [Google Scholar]
  • 66. Moustafa K: No to paid peer review. Lancet. 2022;400(10347):160. 10.1016/S0140-6736(22)01057-1 [DOI] [PubMed] [Google Scholar]
  • 67. Squazzoni F, Bravo G, Takacs K: Does incentive provision increase the quality of peer review? An experimental study. Res. Policy. 2013;42(1):287–294. 10.1016/j.respol.2012.04.014 [DOI] [Google Scholar]
  • 68. Bonaccorsi A: Towards Peer Review As a Group Engagement. JLIS. It. 2022;14(1):46–59. 10.36253/jlis.it-511 [DOI] [Google Scholar]
  • 69. Chang J-J, Lai C-C: Is It Worthwhile to Pay Referees? South. Econ. J. 2001;68(2):457–463. [Google Scholar]
  • 70. Bonaccorsi A: Towards Peer Review As a Group Engagement. JLIS. It. 2022;14(1):46–59. 10.36253/jlis.it-511 [DOI] [Google Scholar]
  • 71. Johnson R: Not-for-profit scholarly publishing might not be cheaper – And that’s OK. LSE. 2024; https://blogs.lse.ac.uk/impactofsocialsciences/2024/01/09/not-for-profit-scholarly-publishing-might-not-be-cheaper-and-thats-ok/. [Google Scholar]
  • 72. Copiello S: On the money value of peer review. Scientometrics. 2018;115:613–620. 10.1007/s11192-018-2664-3 [DOI] [Google Scholar]
  • 73. Beaufils P, Karlsson J: Legitimate division of large datasets, salami slicing and dual publication. Where does a fraud begin? Orthop. Traumatol. Surg. Res. 2013;99(2):121–122. 10.1016/j.otsr.2013.01.001 [DOI] [PubMed] [Google Scholar]
  • 74. Rawat S, Meena S: Publish or perish: Where are we heading? J. Res. Med. Sci. 2014;19(2):87–89. [PMC free article] [PubMed] [Google Scholar]
  • 75. Yeo MA, Renandya WA, Tangkiengsirisin S: Re-envisioning Academic Publication: From “Publish or Perish” to “Publish and Flourish”. RELC J. 2022;53(1):266–275. 10.1177/0033688220979092 [DOI] [Google Scholar]
  • 76. Ioannidis JPA, Klavans R, Boyack KW: Thousands of scientists publish a paper every five days. Nature. 2018;561(7722):167–169. 10.1038/d41586-018-06185-8 [DOI] [PubMed] [Google Scholar]
  • 77. Ioannidis JPA, Collins TA, Baas J: Evolving patterns of extremely productive publishing behavior across science. 2023. Preprint. 10.1101/2023.11.23.568476v2 [DOI]
  • 78. Van Noorden R: The science that’s never been cited. Nature. 2017;552(7684):162–164. 10.1038/d41586-017-08404-0 [DOI] [PubMed] [Google Scholar]
  • 79. Arns M: Open access is tiring out peer reviewers. Nature. 2014;515(7528):467. 10.1038/515467a [DOI] [PubMed] [Google Scholar]
  • 80. Rooyen S, Godlee F, Evans S, et al. : Effect of blinding and unmasking on the quality of peer review: a randomized trial. JAMA. 1998;280(3):234–237. 10.1001/jama.280.3.234 [DOI] [PubMed] [Google Scholar]
  • 81. Kennison R: Back to the future: (re) turning from peer review to peer engagement. Learn. Publ. 2016;29(1):69–71. 10.1002/leap.1001 [DOI] [Google Scholar]
  • 82. Ross-Hellauer T: What is open peer review? A systematic review [version 2; peer review: 4 approved]. F1000Res. 2017;6:588. 10.12688/f1000research.11369.1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 83. Marcoci A, Vercammen A, Bush M, et al. : Reimagining peer review as an expert elicitation process. BMC. Res. Notes. 2022;15(1):127. 10.1186/s13104-022-06016-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 84. Bell GP, Kvajo M: Tackling waste in publishing through portable peer review. BMC Biol. 2018;16(1):146. 10.1186/s12915-018-0619-z [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 85. Guillemaud T, Facon B, Bourguet D: Peer Community In: A free process for the recommendation of unpublished scientific papers based on peer review, in ELPUB 2019 23rd edition of the International Conference on Electronic Publishing.2019: Marseille, France. [Google Scholar]
  • 86. Spezi V, Wakeling S, Pinfield S, et al. : Open-Access mega-journals: The future of scholarly communication or academic dumping ground? A review. J. Doc. 2017;73:263–283. 10.1108/JD-06-2016-0082 [DOI] [Google Scholar]
  • 87. Ling F: Improving peer review: increasing reviewer participation. Learn. Publ. 2011;24:231–233. 10.1087/20110311 [DOI] [Google Scholar]
  • 88. Papatriantafyllou M: Peer Review - the future is here. FEBS Lett. 2017;591(18):2789–2792. 10.1002/1873-3468.12792 [DOI] [PubMed] [Google Scholar]
  • 89. Kousha K, Thelwall M: Artificial intelligence to support publishing and peer review: A summary and review. Learn. Publ. 2024;37:4–12. (in press). 10.1002/leap.1570 [DOI] [Google Scholar]
  • 90. Heaven D: AI peer reviewers unleashed to ease publishing grind. Nature. 2018;563(7733):609–610. 10.1038/d41586-018-07245-9 [DOI] [PubMed] [Google Scholar]
  • 91. Graur D: Payback time for referee refusal. Nature. 2014;505:483. 10.1038/505483a [DOI] [PubMed] [Google Scholar]
  • 92. Laxdal A, Haugen T: Where are the carrots? A proposal to start crediting peer reviewers for their contribution to science. Learn. Publ. 2024;37:154–156. (in press). 10.1002/leap.1589 [DOI] [Google Scholar]
  • 93. Gorin MA: Combating reviewer fatigue with carrots. BJUI Compass. 2023;4(1):3–4. 10.1002/bco2.207 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94. Kumar MN: The ‘peer reviewer as collaborator’ model for publishing. Learn. Publ. 2010;23(1):17–22. 10.1087/20100105 [DOI] [Google Scholar]
  • 95. Teixeira da Silva JA, Dobranszki J: Problems with traditional science publishing and finding a wider niche for post-publication peer review. Account. Res. 2015;22(1):22–40. 10.1080/08989621.2014.899909 [DOI] [PubMed] [Google Scholar]
  • 96. Teixeira da Silva JA, Al-Khatib A, Dobranszki J: Fortifying the Corrective Nature of Post-publication Peer Review: Identifying Weaknesses, Use of Journal Clubs, and Rewarding Conscientious Behavior. Sci. Eng. Ethics. 2017;23(4):1213–1226. 10.1007/s11948-016-9854-2 [DOI] [PubMed] [Google Scholar]
F1000Res. 2024 Jul 18. doi: 10.5256/f1000research.168408.r300640

Reviewer response for version 2

Farid Rahimi 1

I thank the author for their considerations of my suggestions and for revising the manuscript.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes

Are arguments sufficiently supported by evidence from the published literature?

Partly

Are all factual statements correct and adequately supported by citations?

No

Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Reviewer Expertise:

Cell biology, neurodegeneration, inflammation, COVID-19, academic publishing and peer review.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2024 Jul 10. doi: 10.5256/f1000research.168408.r300638

Reviewer response for version 2

Vinícius Medina Kern 1

Upon reviewing the manuscript v2, I acknowledge that my initial assessment did not identify any major mandatory changes, thus rendering the paper acceptable in its fundamental structure and content. The author has provided an 'Amendments from Version 1' section, which indicates some responsiveness to the feedback received.

However, I must note with concern that the 'Minor Comments' from my initial review have largely been disregarded. These comments, though seemingly minor, are crucial for enhancing the clarity and readability of the paper. For instance, the citation format currently resembles the APA style, which may mislead readers into believing that the paper by Waltman (2023) is a sole authorship, whereas it is not. This discrepancy needs to be addressed, either by adhering strictly to the APA style or clearly indicating the use of an alternative referencing standard.

Moreover, there are other minor yet significant issues highlighted in my initial review that remain unresolved. Ignoring these minor corrections could disrupt the reading experience and potentially misinform the audience.

In conclusion, while the manuscript is fundamentally sound and makes a valuable contribution to the field, I recommend approval with restrictions. The author should address the minor corrections outlined in my initial review to ensure the manuscript meets the high standards expected by our readership.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes

Are arguments sufficiently supported by evidence from the published literature?

Yes

Are all factual statements correct and adequately supported by citations?

Yes

Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Reviewer Expertise:

educational peer review, scholarly communication, research evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

F1000Res. 2024 Jul 2. doi: 10.5256/f1000research.163380.r282628

Reviewer response for version 1

Vinícius Medina Kern 1

This opinion paper makes a well-argued case in favor of paying referees. The article acknowledges both the literature in support of and against the argument.

Major Comments:

The case is parsimoniously introduced in the section "No more motivation to review for free", where it states that "monetary incentives might be one potential means to alleviate the burden on the peer review process to some extent" and that "paying professional reviewers can be implemented in conjunction with any of the solutions below" (in Table 1). Table 1 is an interesting summary of various kinds of proposed solutions to improve peer review.

"Peer review cannot thrive in the current academic ecosystem" is a fortunate name for the section before Conclusion. While I think it is enough for this paper, it only touches the tip of the iceberg. The literature on peer review is rich, but it is scarce in describing and explaining peer review as a functioning system. The current system appears to serve commercial publishers, who profit significantly from it, far better than it does authors, referees, and often under-compensated editors. The components, environment, and interconnections that underpin the system and drive its dynamics require further exploration. The Open Access movement has successfully promoted free readership, which seems good and logical given the public funding behind most research. However, it has not succeeded in reducing the costs exacerbated by the Serials Crisis, that motivated the movement.

Furthermore, the peer review system has historically failed the research assessment culture by keeping reviews hidden from all but the editor, the author, and the referees (except for their own reviews). This has resulted in decades of systemic non-learning in paper assessment. The Open Science Movement now advocates for open reviews, a commendable initiative, though not always feasible for several reasons. The lack of shared reviews among referees over the decades remains a significant shortcoming. Making everything open is quite a radical change. This is clearly not a critique of this paper but highlights a broader systemic issue.

Regarding potential hindrances to implementing reviewer payment, I suggest the author consider mentioning the fact that the proposed solution is hardly feasible for journals in Diamond Open Access - the ones that do not charge from author nor readers, mostly from poorer countries. They mostly rely on voluntary work, with some support from universities or government funding.

Minor Comments:

The correct citation for item 18 ("according to Waltman(2023)") seems to be "according to Waltman et al. (2023)."

Some editing might improve readability. For instance, in "Those qualities, promoted by other schools of thoughts, are ...", my understanding halted so I could think if the school of thought adopted also promotes those qualities and if the other schools of thought are the three other ones described by Waltman et al. (2023). It would be clear to me and the disturbance would not occur if the segment was written as "Those qualities, also promoted by the other schools of thought, are ..."

The article effectively uses the current literature, but I feel some statements lack a citation nonetheless. For instance, "... increase in instances of fraud where for example false contact information of potential referees with non-institutional emails are suggested or when the peer review process has been manipulated" could be backed by sources such as [1] or [2] (for instance).

I think it would be appropriate to mention Publons (Table 1, p. 6) pointing out that it was bought by Clarivate and does not exist under that name, or, at least, to give a link (http://webofscience.help.clarivate.com/Content/publons.html). Future readers might be unable to find out what Publons was.

In Figure 1, the blue and green boxes are difficult to distinguish. They appeared identical to me - until I read the figure label.

There's are typos that I suppose either the author of the editorial team will correct, such as "no through evaluation of the reviewed papers" ("thorough") and "Is the ultimate goal of academia to produce ... sometimes with questionable practices." (missing interrogation mark).

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes

Are arguments sufficiently supported by evidence from the published literature?

Yes

Are all factual statements correct and adequately supported by citations?

Yes

Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Reviewer Expertise:

educational peer review, scholarly communication, research evaluation

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

References

  • 1. : Retractions, Fake Peer Reviews, and Paper Mills. J Korean Med Sci .2021;36(24) : 10.3346/jkms.2021.36.e165 e165 10.3346/jkms.2021.36.e165 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2. Paying reviewers and regulating the number of papers may help fix the peer-review process. https://www.researchgate.net/publication/358423532_Exploratory_analysis_of_text_duplication_in_peer-review_reveals_peer-review_fraud_and_paper_mills/citation/download .2022; 10.1007/s11192-022-04504-5 10.1007/s11192-022-04504-5 [DOI] [PMC free article] [PubMed]
F1000Res. 2024 Jun 21. doi: 10.5256/f1000research.163380.r282627

Reviewer response for version 1

Farid Rahimi 1

Dr Seghier has written a comprehensive narrative opinion piece on the peer-review process as it is an integral part of the academic ecosystem. Dr Seghier has highlighted the fundamental issues that challenge(d) the efficacy and efficiency of the peer-review process contemporarily (or historically) and argues for establishing a transparent system whereby peer reviewers could be financially rewarded and recompensed. De Seghier argues that capping the number of paper submissions may also help reform the peer-review process.

I have listed my comments that I hope will help improve the opinion piece or open up the perspective to discussion. The manuscript is written relatively well; however, summarising the text will benefit the readers.

Major comments:

I suggest adding the emergence of competing mega-journals or mega-publishers as another issue ‎that may affect the peer-review process (in the Abstract). This was mentioned in Table 1 and in reference 70.‎ I believe this is part of the problem.

The last statement of the Abstract contradicts the argument for reform. Please revise “Instead of ‎reforming the peer-review process …” because the piece calls for reform.‎

Change the somehow hypothetical, “It can exist …” to factual, “It exists …” or “Peer review is ‎performed in different ways.” ‎

Please note that the description of the traditional roles of academics includes or represents the mainstream tenured academics. However, ‎peer-reviewing is undertaken by non-tenured staff and academics with relevant ‎qualifications, experience, and expertise. Reviewers in this category (for example, postdoctoral ‎scholars, senior technical officers, qualified research assistants, qualified laboratory managers, ‎or PhD-qualified persons who have quit academia, qualified academics who are active in postgraduate teaching) undertake peer-review merely to serve the ‎scientific community; however, they generally cannot claim academic promotions for their voluntary participation in ‎the peer-review process (because they do not have grants or tenured positions in a faculty). Thus, financially rewarding ‎them makes perfect sense for their service and inclusion.‎ Such an understanding acknowledges the inclusivity of the peer review process.

Similarly, the suggestion of co-authorship and academic benefits and promotions is interesting as ‎presented in the Table 1. However, this excludes those reviewers who are not ‎tenured academics (as discussed above). ‎Please revise the paragraph 2 in page 7 of 13 (PDF file) to remove the assumption or assertion that all reviewers ‎are academicians. ‎

Please note that the remuneration of $200 (in Table 1 and page 8 of 13) or $450 are ‎prescriptive. Some reviewing tasks demand many valuable hours if done diligently. Larger pieces of work and larger manuscripts with complex data and messages will require more time for thorough assessment. A mechanism ‎for determining this remuneration should be discussed instead of prescribing a fixed amount (many factors may interplay). ‎Determining the remuneration may not be simple as suggested or alluded by the author in the ‎context of low-income countries.‎ If this is left with the publishers, or some publishers take on this initiative or control, the publication charges and profit reaping will increase by many folds. What body controls this is unimaginable.

Please revise the assumptions that the funding bodies cover the publication charges and how ‎they should work closely with the publishers to establish a fair means of paying the reviewers. ‎This is confusing to me because, as I understand, funding bodies provide the grants to the academics ‎or the researchers. The latter then need to budget for publication charges out of their granted ‎funds. I also understand that the funding bodies normally do not interact with the publishers that ‎publish a funded project that was conducted by the funded researchers or academics. Funding bodies also do not interfere with the research plans or outcomes. ‎

Please note that Publons does not exist anymore. Publons was a commercial website that ‎provided a free service for academics to track, verify, and showcase their peer review and ‎editorial contributions to academic journals. It was launched in 2012 but was bought by Clarivate ‎Analytics in 2017. https://en.wikipedia.org/wiki/Publons.‎

Minor comments:

The title is wordy. Please consider rephrasing/summarizing, e.g., “Paying academic reviewers and regulating the number of submitted papers may help fix the peer-review process.” Or “Remunerating peer reviewers and restricting the number of submitted papers may help fix the peer-‎review process.‎”

Please rewrite “It might be useful to contemplate what could be the alternative if no adequate and sustainable solutions are found.” to remove the dummy subject “it” and the hypothetical nature of the statement and clarify the message. For example, “Finding adequate and sustainable alternative solutions will be useful [or “is needed”].”

Please revise and summarize the following sentence and juxtapose “include or including” with the roles or duties. For example, “the traditional job of academia is being transformed with extra roles that researchers are expected to assume, leaving no time for reviewing papers, including admin duties, seeking funds, leadership roles, academic advising, organizing outreach activities in the community, and working on translating their research into marketable solutions,” should be changed to “the roles and duties of the academics include administration, writing grants, writing papers, writing patents, commercialization ventures, leaderships and directorships, editorships, teaching, supervision, committee memberships, consultancy, and outreach, leaving no time for reviewing papers.”

Please change “in other languages than English” to “in languages other than English”.‎

Please summarize the following statement to remove repetitions: “researchers are increasingly ‎unsatisfied with for-profit publishers that have made exorbitant profits with negligible contribution to ‎research in general, resulting in many researchers not keen to review for for-profit journals”. For ‎example, “academics and authors avoid peer-reviewing because they are increasingly unsatisfied ‎with for-profit publishers that reap exorbitant ‎profits while contribute negligibly towards academia and research.” While mentioning this, some journals or publishers sponsor conferences or provide thesis or paper prizes and competitions. However, the proportion of this contribution could be small. I’m not sure if this has been studied systematically. ‎

In Table 1, revise the following:

“basically based on”

“Reviewers time” to “Reviewers’ time”

“Add some delays to the publication of papers” to “Delays the publication”

“Reviewers’ databases”

“to oversight” to “to oversee”

“Augment the peer process” to “Augment the review process”

“are not Involved” to “are not involved”

Rephrase the subtitle, “Toward the promotion of Professional Reviewer as a career” to “Promoting professional reviewing as a career”. A review cannot be factually (logically) promoted as a career.

Please rephrase sentences that use dummy subjects, “it is,” “there is,” or “there are” as suggested by the principles of plain English and for the sake of brevity.

Please avoid personifications, for example, “… this paper reckons …” while the credit should go to the author.

Avoid using the passive voice and nominalizations (using verbs as nouns requiring helper verbs). For example, “Many calls have been made to reform …”

Avoid contractions in academic writing, e.g., “I’m.”

Correct the minor spelling mistakes:

‘chose’ [choose in two places]

‘through evaluation’ [thorough evaluation]

between journals [among journals]

add a question mark after ‘Is the ultimate …’,

‘will soon or later’ [will sooner or later] …

The paper will benefit from proofreading, editing, and summarization.

Thank you for the opportunity to review your work.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes

Are arguments sufficiently supported by evidence from the published literature?

Partly

Are all factual statements correct and adequately supported by citations?

No

Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Reviewer Expertise:

Cell biology, neurodegeneration, inflammation, COVID-19, academic publishing and peer review.

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

F1000Res. 2024 Jun 25.
Mohamed Seghier 1

Point-by-point responses:

I would like to thank both reviewers for their very helpful and constructive comments. All their concerns have been addressed in this resubmitted revision.

All changes are highlighted in the revised manuscript with track changes.

Below, is a detailed point-by-point rebuttal addressing each reviewer’s comment.

Reviewer #2: Farid Rahimi, Australian National University, Canberra, Australian Capital Territory, Australia.

Dr Seghier has written a comprehensive narrative opinion piece on the peer-review process as it is an integral part of the academic ecosystem. Dr Seghier has highlighted the fundamental issues that challenge(d) the efficacy and efficiency of the peer-review process contemporarily (or historically) and argues for establishing a transparent system whereby peer reviewers could be financially rewarded and recompensed. De Seghier argues that capping the number of paper submissions may also help reform the peer-review process. I have listed my comments that I hope will help improve the opinion piece or open up the perspective to discussion. The manuscript is written relatively well; however, summarising the text will benefit the readers.

Thank you very much for your positivity and the very useful suggestions.

Reviewer #2, continued:

Major comments:

I suggest adding the emergence of competing mega-journals or mega-publishers as another issue ‎that may affect the peer-review process (in the Abstract). This was mentioned in Table 1 and in reference 70.‎ I believe this is part of the problem.

Response:

Thank you for this very interesting suggestion. I have briefly mentioned this issue in the new version of the Abstract: “ the emergence of competing mega-journals”.

Reviewer #2, continued:

The last statement of the Abstract contradicts the argument for reform. Please revise “Instead of ‎reforming the peer-review process …” because the piece calls for reform.‎

Response:

I fully agree with the reviewer. Accordingly, I have deleted that statement from the Abstract.

Reviewer #2, continued:

Change the somehow hypothetical, “It can exist …” to factual, “It exists …” or “Peer review is ‎performed in different ways.” ‎

Response:

Done! Thank you.

Reviewer #2, continued:

Please note that the description of the traditional roles of academics includes or represents the mainstream tenured academics. However, ‎peer-reviewing is undertaken by non-tenured staff and academics with relevant ‎qualifications, experience, and expertise. Reviewers in this category (for example, postdoctoral ‎scholars, senior technical officers, qualified research assistants, qualified laboratory managers, ‎or PhD-qualified persons who have quit academia, qualified academics who are active in postgraduate teaching) undertake peer-review merely to serve the ‎scientific community; however, they generally cannot claim academic promotions for their voluntary participation in ‎the peer-review process (because they do not have grants or tenured positions in a faculty). Thus, financially rewarding ‎them makes perfect sense for their service and inclusion.‎ Such an understanding acknowledges the inclusivity of the peer review process. Similarly, the suggestion of co-authorship and academic benefits and promotions is interesting as ‎presented in the Table 1. However, this excludes those reviewers who are not ‎tenured academics (as discussed above). ‎Please revise the paragraph 2 in page 7 of 13 (PDF file) to remove the assumption or assertion that all reviewers ‎are academicians. ‎

Response:

Thank you for raising this important point. Accordingly, I have made the following changes:

To Table 1: I have added this brief statement: “ However, this model might exclude those reviewers who are not tenured academics. ”.

To Page 6, the edited text now reads: “ Furthermore, peer-reviewing is also undertaken by non-tenured staff and academics with relevant qualifications and expertise (i.e. postdocs, senior technical staff, research assistants, laboratory engineers and managers, and PhD-qualified persons who have quit academia). However, they generally cannot claim academic promotions for their voluntary participation in the peer-review process. Thus, an inclusive peer review process must acknowledge their contributions by financially rewarding them.

Reviewer #2, continued:

Please note that the remuneration of $200 (in Table 1 and page 8 of 13) or $450 are ‎prescriptive. Some reviewing tasks demand many valuable hours if done diligently. Larger pieces of work and larger manuscripts with complex data and messages will require more time for thorough assessment. A mechanism ‎for determining this remuneration should be discussed instead of prescribing a fixed amount (many factors may interplay). ‎Determining the remuneration may not be simple as suggested or alluded by the author in the ‎context of low-income countries.‎ If this is left with the publishers, or some publishers take on this initiative or control, the publication charges and profit reaping will increase by many folds. What body controls this is unimaginable.

Response:

Thank you for raising this important point. The two figures ($200 and $450) were in fact suggested in prior work. I agree that many factors influence the level of remuneration, and I don’t think that a single mechanism can determine the exact remuneration for so many journals in very diverse domains. Having said that, I have added the following paragraph to Page 7:

Many factors may interplay to determine the level of remuneration [45, 54, 55], including reviewing for not-for-profit versus for-profit journals, scientific domains (e.g. research in medicine is more funded and expensive than research in art and humanities), the number of reviewers required per manuscript (e.g. two or more referees), the type of articles (e.g. reviewing a research article requires more hours than a commentary), and their length and complexity (e.g. reviewing a long review article or a full research article with complex data is more demanding than an opinion piece). Publishers and editors should estimate the level of remuneration according to the average hours typically spent by referees for each article type [45] and the average yearly number of manuscripts reviewed by each reviewer [36, 55]. Other organizations and agencies can also establish general guidelines about the payment of reviewers, including the Committee on Publication Ethics, the Declaration on Research Assessment, the Confederation of Open Access Repositories, and the European Association of Science Editors.

Reviewer #2, continued:

Please revise the assumptions that the funding bodies cover the publication charges and how ‎they should work closely with the publishers to establish a fair means of paying the reviewers. ‎This is confusing to me because, as I understand, funding bodies provide the grants to the academics ‎or the researchers. The latter then need to budget for publication charges out of their granted ‎funds. I also understand that the funding bodies normally do not interact with the publishers that ‎publish a funded project that was conducted by the funded researchers or academics. Funding bodies also do not interfere with the research plans or outcomes. ‎

Response:

Thank you for allowing me to clarify this statement. Accordingly, I have edited that paragraph as below (see Page 7):

Publications are typically included as deliverables in research grants, and therefore the contribution of peer reviewers to improve these deliverables can be budgeted under the category of consultancy fees. Funding bodies may categorize this type of expense as publication fees. However, regardless of how paying reviewers is included in the budget, too much financial burden will be put on funding bodies as they are already covering the excessive article-processing charges. As alternatives are being sought for article-processing charges [56], funding bodies should put in place guidelines to make the payment process of professional reviewers as fair and transparent as possible, in particular when for-profit agencies are managing the peer review process. For instance, many funding bodies already require that published work must be made freely available, and thus similar requirements can also be made about the type of peer review (free or paid). If there is strong evidence that paying reviewers improves the quality of publications, then funding bodies can recommend researchers to publish in journals that pay referees. This ensures that funded research is being evaluated and scrutinized by the most rigorous review process that scholarly communication can offer (and afford).

Reviewer #2, continued:

Please note that Publons does not exist anymore. Publons was a commercial website that ‎provided a free service for academics to track, verify, and showcase their peer review and ‎editorial contributions to academic journals. It was launched in 2012 but was bought by Clarivate ‎Analytics in 2017. https://en.wikipedia.org/wiki/Publons.‎

Response:

Although the brand name Publons ceased to exist, some of its functionalities are still available via Web of Science. To avoid any confusion, I have now deleted it from Table 1.

Reviewer #2, continued:

Minor comments:

The title is wordy. Please consider rephrasing/summarizing, e.g., “Paying academic reviewers and regulating the number of submitted papers may help fix the peer-review process.” Or “Remunerating peer reviewers and restricting the number of submitted papers may help fix the peer-‎review process.‎”

Response:

As suggested by the reviewer, I have now added the following new title: “ Paying reviewers and regulating the number of papers may help fix the peer-review process”.

Reviewer #2, continued:

Please rewrite “It might be useful to contemplate what could be the alternative if no adequate and sustainable solutions are found.” to remove the dummy subject “it” and the hypothetical nature of the statement and clarify the message. For example, “Finding adequate and sustainable alternative solutions will be useful [or “is needed”].”

Response:

This statement has been rewritten as recommended by the reviewer; see Page 3.

Reviewer #2, continued:

Please revise and summarize the following sentence and juxtapose “include or including” with the roles or duties. For example, “the traditional job of academia is being transformed with extra roles that researchers are expected to assume, leaving no time for reviewing papers, including admin duties, seeking funds, leadership roles, academic advising, organizing outreach activities in the community, and working on translating their research into marketable solutions,” should be changed to “the roles and duties of the academics include administration, writing grants, writing papers, writing patents, commercialization ventures, leaderships and directorships, editorships, teaching, supervision, committee memberships, consultancy, and outreach, leaving no time for reviewing papers.”

Response:

Thank you for this suggestion. I have added it to Page 5 of the new revised version.

Reviewer #2, continued:

Please change “in other languages than English” to “in languages other than English”.‎

Response:

Done! Thank you.

Reviewer #2, continued:

Please summarize the following statement to remove repetitions: “researchers are increasingly ‎unsatisfied with for-profit publishers that have made exorbitant profits with negligible contribution to ‎research in general, resulting in many researchers not keen to review for for-profit journals”. For ‎example, “academics and authors avoid peer-reviewing because they are increasingly unsatisfied ‎with for-profit publishers that reap exorbitant ‎profits while contribute negligibly towards academia and research.” While mentioning this, some journals or publishers sponsor conferences or provide thesis or paper prizes and competitions. However, the proportion of this contribution could be small. I’m not sure if this has been studied systematically. ‎

Response:

I have replaced that statement with the reviewer’s suggestion (see Page 5 of the new revised version).

Regarding your second point, I’m not aware of any systematic analysis of the contribution of for-profit publishers to scholarly communication. While they do sponsor some academic events, I believe this is a small fraction of what they get as profits (e.g. according to some estimations, Elsevier for instance has a 40% profit margin on its $2 Billion annual revenue). I did not mention this point in the revised version because it is extremely hard to get the exact figures when it comes to the money made or spent by publishers.

Reviewer #2, continued:

In Table 1, revise the following:

“basically based on”

“Reviewers time” to “Reviewers’ time”

“Add some delays to the publication of papers” to “Delays the publication”

“Reviewers’ databases”

“to oversight” to “to oversee”

“Augment the peer process” to “Augment the review process”

“are not Involved” to “are not involved”

Response:

Done! Thank you for being meticulous here.

Reviewer #2, continued:

Rephrase the subtitle, “Toward the promotion of Professional Reviewer as a career” to “Promoting professional reviewing as a career”. A review cannot be factually (logically) promoted as a career.

Response:

Thank you for your suggestion. I have changed the subheading accordingly.

Reviewer #2, continued:

Please rephrase sentences that use dummy subjects, “it is,” “there is,” or “there are” as suggested by the principles of plain English and for the sake of brevity.

Response:

Thank you. I have removed many instances of “it is” and “there is”.

Reviewer #2, continued:

Please avoid personifications, for example, “… this paper reckons …” while the credit should go to the author.

Response:

This is now replaced by “I reckon …”. Thank you

Reviewer #2, continued:

Avoid using the passive voice and nominalizations (using verbs as nouns requiring helper verbs). For example, “Many calls have been made to reform …”

Response:

I have now reduced the occurrence of passive forms (has/have been). Thank you.

Reviewer #2, continued:

Avoid contractions in academic writing, e.g., “I’m.”

Response:

Done! Thank you.

Reviewer #2, continued:

Correct the minor spelling mistakes:

‘chose’ [choose in two places]

‘through evaluation’ [thorough evaluation]

between journals [among journals]

add a question mark after ‘Is the ultimate …’,

‘will soon or later’ [will sooner or later] …

Response:

Done! Thank you.

Reviewer #2, continued:

The paper will benefit from proofreading, editing, and summarization.

Thank you for the opportunity to review your work.

Response:

I have edited the manuscript to correct any typing or grammatical errors. Thank you again for all these very useful and constructive comments and suggestions.

F1000Res. 2024 May 31. doi: 10.5256/f1000research.163380.r277016

Reviewer response for version 1

Balazs Aczel 1

All in all, I found that the manuscript discusses an important and timely topic along a well structured argument. 

My only major issue with the message of the paper is that it proposes paying referees as a solution to the peer review crisis although there are a number of listed and unresolved issues with the ideas. For example, the author shortly discusses that if paying reviewers becomes a norm then they might become uninterested in reviewing for free. This would be a very negative development for not-for-profit publishers, further strengthening the for-profit model that the author criticises. The author suggests that the budget for paying the reviewers could come from $200 submission fees which could be reduced from the article processing charges. Although it sounds like a fair solution, it would go against the publishers' financial interest since they would "lose" that income. From a profit-oriented point-of-view, it either makes the proposal sound unsavory unless the loss can be compensated by further increasing the APCs.

Minor issues:

- " exponential increase in the number of submissions", "desk rejection rates in reputable journals are reaching high rates" are factual statements without reference to evidence.

- Among the potential solution could be mentioned PCI https://peercommunityin.org/ where the community organizes the reviews without any involvement of the journals. This model should reduce redundancies in the peer review system.

Is the topic of the opinion article discussed accurately in the context of the current literature?

Yes

Are arguments sufficiently supported by evidence from the published literature?

Yes

Are all factual statements correct and adequately supported by citations?

Yes

Are the conclusions drawn balanced and justified on the basis of the presented arguments?

Yes

Reviewer Expertise:

metascience

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

F1000Res. 2024 Jun 25.
Mohamed Seghier 1

Point-by-point responses:

I would like to thank both reviewers for their very helpful and constructive comments. All their concerns have been addressed in this resubmitted revision.

All changes are highlighted in the revised manuscript with track changes.

Below, is a detailed point-by-point rebuttal addressing each reviewer’s comment.

Reviewer #1: Balazs Aczel, Eotvos Lorand University, Budapest, Hungary.

All in all, I found that the manuscript discusses an important and timely topic along a well-structured argument.

Thank you very much for your positivity. I’m happy to read that you did find the current version important and well-structured.

Reviewer #1, continued:

My only major issue with the message of the paper is that it proposes paying referees as a solution to the peer review crisis although there are a number of listed and unresolved issues with the ideas. For example, the author shortly discusses that if paying reviewers becomes a norm then they might become uninterested in reviewing for free. This would be a very negative development for not-for-profit publishers, further strengthening the for-profit model that the author criticises.

Response:

Thank you for raising this important point. I have now added the following statement to clarify this issue on Page 10: “ There is also the risk that paying referees will soon become the norm, making researchers no longer interested in reviewing for free. It could also lead to a multi-tier peer review system depending on how much a journal chooses to pay its reviewers, hence engendering an unequal competition among journals for the service of reviewers. This could hurt not-for-profit publishers that cannot afford an expensive peer review process. In that context, not-for-profit publishers must (i) diversify incentives on top of paying reviewers, (ii) nurture the intrinsic motivation of their readers to participate in peer review and serve their scientific community [67] , (iii) seek alternative means to be subsidized [69] , and (iv) foster strategic partnerships with all stakeholders to make not-for-profit journals the preferred model for open-access publicly-funded research. Overall, although one cannot rule out the risk of corruption in peer review by money, I trust that academia has built-in mechanisms to handle financial transactions fairly and objectively, similar to cases when researchers are paid to evaluate grant proposals and academic theses [53] .

Reviewer #1, continued:

The author suggests that the budget for paying the reviewers could come from $200 submission fees which could be reduced from the article processing charges. Although it sounds like a fair solution, it would go against the publishers' financial interest since they would "lose" that income. From a profit-oriented point-of-view, it either makes the proposal sound unsavory unless the loss can be compensated by further increasing the APCs.

Response:

I agree with the reviewer here. Accordingly, I have made the following changes:

To Page 8, I have added: “ However, this might sound unsavory for for-profit publishers and hence the community needs to create alternative models that would make publishers active contributors to peer review, as discussed below.

See also the new edited text in the section “ What about the publishers?” (Page 9-10).

Reviewer #1, continued:

Minor issues:

- " exponential increase in the number of submissions", "desk rejection rates in reputable journals are reaching high rates" are factual statements without reference to evidence.

Response:

Thank you for spotting these missing citations. I have now added the following references:

Bornmann L, Haunschild R, Mutz R (2021) Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases. Humanities and Social Sciences Communications, 8:224.

Larsen PO, von Ins M (2010) The rate of growth in scientific publication and the decline in coverage provided by Science Citation Index. Scientometrics, 84:575-603. 10.1007/s11192-010-0202-z.

Meyer HS, Durning SJ, Sklar DP, Maggio LA (2018) Making the First Cut: An Analysis of Academic Medicine Editors' Reasons for Not Sending Manuscripts Out for External Peer Review. Acad Med, 93:464-470. 10.1097/ACM.0000000000001860.

Reviewer #1, continued:

- Among the potential solution could be mentioned PCI https://peercommunityin.org/ where the community organizes the reviews without any involvement of the journals. This model should reduce redundancies in the peer review system.

Response:

Thank you for this very important suggestion. I have now added PCI to Table 1. Please see the new revised version of the resubmitted manuscript.

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Data Availability Statement

    No data are associated with this article.


    Articles from F1000Research are provided here courtesy of F1000 Research Ltd

    RESOURCES