Peer review under review: room for improvement?

The peer review process is a central part of medicine and has become a touchstone of modern evaluation of scientific quality. Although generally considered essential to academic quality, peer review has been increasingly criticised as ineffective, slow, and misunderstood. A frequent claim is that the process is insufficiently objective and that it is inconsistent in its capacity to assess manuscript quality. Implicit in this is the expectation that manuscript review is, or should be, a ‘scientific’ process, but it should be realised that manuscript review cannot be a ‘scientific’ process. This needs to be taken into account both by those carrying out reviews and those who wish to evaluate and improve the manuscript review process.

Peer review requires a community of experts in a given field, who are qualified and able to perform impartial review. Impartial review, especially of work in less narrowly defined or inter-disciplinary fields, may be difficult to accomplish; the significance (good or bad) of an idea may never be widely appreciated among its contemporaries. Richard Horton, editor-in-chief of The Lancet, has stated that ‘peer review to the public is portrayed as a quasisacred process that helps to make science our most objective truth teller, but we know that the system of peer review is biased, unjust, unaccountable, incomplete, easily fixed, often insulting, usually ignorant, occasionally foolish, and frequently wrong’. Publication bias and discrimination are increasingly recognised in medicine. A recent survey showed that medical journals were almost three times more likely to publish reports from their own editorial board members than from one of the rival's journals within their subspeciality. It has also been shown that reviewers are biased when they allegedly knew or suspected the author's identity. In this respect, there might be a huge conflict of interest between reviewers and authors when they work in the same field but belong to different research groups, competing for the same grants and subsequent prestige. Double-masking (both reviewers and authors are anonymous to each other) may improve the quality of biomedical publishing or at least reduce reviewer bias for effectively masked manuscripts, but this is still subject to controversy. The interposition of reviewers between authors and readers offers the possibility that reviewers may serve as gatekeepers. Peer review is susceptible to control by experts and to personal jealousy. It may suppress ‘dissent’ against ‘mainstream’ theories. Reviewers tend to be especially critical of opinions that contradict their own views, and they are therefore lenient towards those that accord with them. Ideas that harmonise with those of the expert opinion leaders are more likely to see print and to appear in premier journals than are revolutionary ones.

Consequently, peer review has become the subject of two extreme visions; on one hand, it is considered as ‘a non-validated charade whose processes generate results little better than does chance’¹; on the other hand ‘it is one of the sacred pillars of the scientific edifice and a necessary condition in quality assurance for scientific publications’.² In order to have the peer review process ‘peer-reviewed’, an International Symposium on Peer Reviewing: ISPR (httpwww.ICTconfer.orgispr) is being organised in the context of The 3rd International Conference on Knowledge Generation, Communication and Management: KGCM 2009, which will be held on 10–13 July 2009, in Orlando, Florida, USA. In this symposium advantages and disadvantages of the peer review process will highlighted. Although everyone agrees on peer review as principle, there is a solid disagreement regarding the effectiveness of the methodologies being applied into achieving the objectives implied by the commonly agreed principle. The meeting therefore is intended to identify solutions, innovations and support systems for more effective peer reviewing approaches, models, and methodologies. So there is ample room for improving the peer review process. It is hoped for that the proceedings of the symposium will describe transparent and stringent peer review policies for identifying high scientific quality.