Reply to Bennett et al.: IJROBP study is consistent with our findings and offers insights on author preferences

In a letter (1) written in response to our PNAS article on single-blind versus double-blind reviewing (2) the editors of the International Journal of Radiation Oncology • Biology • Physics (IJROBP) describe some interesting results with respect to a question that has been frequently asked of us: the effect of guessing or inferring authorship under a double-blind protocol. These results may be interpreted in the context of other literature on the efficacy of blinding (see, for example, ref. 3, chap. 6).

In the context of clinical trials this phenomenon is known as leakage: To the extent that authors’ identities are known the treatment group comes to resemble the control group, and the full magnitude of the actual treatment effect may be understated. Leakage therefore serves principally to explain findings of nonsignificance. In the case of our study we found significant treatment effects, and the magnitude of these effects is large. In particular, the 22% increase in number of bids from double-blinded reviewers represents substantially more effort than that exhibited by control reviewers with access to author names and affiliations. Thus, in the context of our reported study, if leakage were present it would suggest that the underlying effects were even larger. Of course, this is no guarantee that other conferences and journals would see similar effects.

Bennett et al. (1) also report a favorable survey among authors for double-blind review. This is a new insight to us, and we have not yet made any assessment of author or reviewer happiness. It is, however, consistent with our observation that double-blind reviewers bid on more papers. While increased bidding does not per se connote a positive attitude shift it does document a positive behavioral shift.