At Journal of Food Science (JFS), we take peer review seriously. For all manuscripts, we strive to get 3 good, critical (yet constructive) reviews. We use a single‐anonymized approach, meaning that reviewers know who the authors are, but authors do not know who provided reviews. We have two layers of editorial assessment, Scientific Editors (SEs), who vet each manuscript, deciding if it can go into peer review or not, and Associate Editors (AEs), who solicit expert reviewers and evaluate comments to pass along a decision to the SE. AEs are asked to rate each reviewer for timeliness and quality of their comments.
Could we do better? Perhaps.
A different model of peer review is the double‐anonymized system. I sometimes get invited to provide reviews for articles in other journals where the manuscript has been stripped of any indication of who did the work. This removes any potential bias reviewers might have for specific authors – they judge the work based solely on its scientific merit, not some perceived notion of quality of a lab. We have talked about going to double‐anonymized, but there are potential pitfalls. For example, if the reviewer does not know who wrote the paper, they cannot decide if they have a conflict of interest or not. It is also more work for the staff since they must hide any author attributions.
Another variable is the number of required reviewers. Many journals require “only” two peer reviews. Asking for three is intended to provide a broader assessment of each manuscript, providing more opportunity for different opinions. This is a very good thing, especially in cases where reviewers differ in their opinions.
However, it also means that our AEs have a more difficult task – sometimes finding three qualified reviewers to give feedback is challenging. And it is becoming more and more difficult as our submissions keep going up (likely to exceed 4500 submissions this year). This often tracks with longer times to reach a decision as the AE is continually inviting more and more reviewers, inviting 20 or more potential reviewers to get three to agree to provide comments has become a common situation.
In principle, three reviews should give a broad assessment of a manuscript. Unfortunately, not all reviews are equal. Many reviews are too short or cryptic to be much value to the authors. Periodically, we get review comments that are less constructive than we desire.
For this reason, we ask the AEs to rate each reviewer, which should provide discrimination of good reviewers from poor. However, there are only two categories, one for quality of the review and one for its timeliness (was it within our 14‐day window allotted to reviewers). Quality is rated on a three‐point scale, from excellent (3) to poor (1). Further, despite continuous pleas, the percentage of reviews rated by AEs is only about 50%.
We continually push AEs to rate reviewers, but many ignore our pleas. Several AEs ignore the reviewer rating system completely, never rating a single one. Fortunately, there are a few AEs who rate every single review they receive and several more AEs fall between 85% and 99%. Most are somewhere between the two extremes. We will continue to encourage AEs to rate reviewers so we can track which reviewers may be eligible for promotion to AE.
What else can we do to improve peer review?
Our new Early Career Researchers (ECR) program was put in place just this year to help train up and coming scientists on peer review and editorial work. We selected six candidates from a pool of applicants to participate in the first cohort. Each cohort member was paired with an experienced editor who has helped them work through how to review a manuscript. The second part of the program will train them on making editorial decisions based on peer reviews. Next year, we hope to expand the program, with the long‐term goal of developing a pipeline of trained reviewers and editors to lead JFS into the next generation.
Reviewer recognition is now considered to be an important aspect of publishing, one that could encourage greater participation. We currently have the Outstanding Reviewer Award in honor of Dr. Manfred Kroger for best reviewer in JFS (congratulations to Dr. Jing Zhao again for selection as the 2026 award winner—read the June editorial announcement). We also recognize all reviewers once a year, in the December editorial. But maybe we need more? Anyone have good ideas on how to recognize outstanding reviewers?
Should we pay reviewers? For some reason, this seems to be a question these days; we get such requests regularly, particularly from younger reviewers. I grew up in an era where there is a sense of obligation. If you publish papers in scientific journals, it is your responsibility to give back. That means accepting peer review invitations and giving the best you can at all times, and without expectation of recompense. Maybe that is just my outdated code of conduct. Maybe more importantly, there are serious ethical concerns in paying reviewers to review.
Although reviewers are the core of the peer review system, the AEs play a huge role as well. They evaluate all reviewer comments, synthesize them, and make a judgment that is passed on to the SE. In a real sense, they are also peer reviewers.
Should we have an AE rating system as well, since not all AEs are as diligent as maybe they ought to be? There are numerous potential reasons for why AEs might not be performing as well as expected, not the least being having too little time for this mostly volunteer role. But since we typically promote AEs into open SE roles, it is important to have some sort of assessment scale to distinguish who has earned the next level.
We conduct an annual performance metrics evaluation, but it focuses exclusively on how much time they take to process manuscripts. While this is very important to know, we do not have a consistent way to evaluate the quality of their work across the board. Do they diligently write good notes to authors that support their decision based on the reviewer's remarks? Do they translate their thoughts directly to the SE to help explain their decision? These would help us respond better to authors, particularly of those manuscripts that are rejected.
I will be considering both our peer review policies and potential AE rating systems over the next year. If anyone has suggestions, please feel free to contact me (rwhartel@wisc.edu).
Sincerely,
Rich Hartel, PhD
Editor in Chief, Journal of Food Science
Professor, University of
Wisconsin–Madison