Comment on “Close loop peer review” by Michael Hill

Abstract

Comment on “Close loop peer review” by Michael Hill.

I read with interest the Opinion article by Michael Hill about the potential benefits of linking grant and article reviews in a “closed loop” 1. It is a thought‐provoking idea that deserves serious consideration. I would though like to draw the readers’ attention to a critical difference between the two processes: article reviews can be subjected to quality control, which is the responsibility of the journal editor. Even if this authority is—unfortunately—invoked infrequently, it can be used to weed out arbitrary comments or excessive requirements for revision. The recent trend to publish papers jointly with their reviews, pioneered by EMBO Press journals in 2009, has been perhaps even more instrumental than editorial supervision in safeguarding the quality of article reviews.

The situation is starkly different in regard to the quality of grant reviews. The US National Institutes of Health, which generates upwards of 80,000 reviews annually—more than any other funding agency worldwide—is a prime example. Although the official policy allows applicants to appeal if the review has factual errors or is absent of expertise, and although institute directors can overrule reviews, these actions are either ineffective or practically unheard of, respectively. In reality, the only effective ground for appeal is inappropriate language or demonstrably biased review. There is practically no recourse for objecting to technical errors, unfounded criticism or overlooking of data, however, egregious they may be. I believe that applicants frequently encounter such predicaments, yet are powerless to contest them.

This situation demands the implementation of stringent quality control, perhaps by the chair of the study section and/or other hand‐picked experts. Admittedly, this would increase substantially the effort invested in the review process. Nevertheless, I believe that what is at stake—the future of the applicants’ careers, fully deserves it.

EMBO Reports (2019) 20: e48313