Journal of Psychiatry and Neuroscience peer review process, with advice for authors

A few years ago I wrote an editorial discussing some of the research that influences how the peer review process is carried out at the Journal of Psychiatry and Neuroscience (JPN).1 It gave the rationale for using nonblinded review by reviewers who remain anonymous to the authors. As discussed, inter-rater reliability in peer review is poor. However, it remains the least bad of the various methods available to maximize the possibility that published manuscripts are the best of those submitted to the journal and that they contain valid results and a well-balanced discussion. This editorial discusses the more practical aspects of peer review — a topic that excites emotion. By describing the details of the peer review process, I hope that as an editor I can help lessen the aggravation that authors, and occasionally reviewers, sometimes feel.

At the time of writing, I am editor-in-chief but soon should be 1 of 2 coeditors-in-chief. With 2 of us, the assignment of manuscripts takes into account our various areas of expertise and evens out the workload. A request to have a manuscript dealt with by one or the other will be honoured if a legitimate reason is given. Obviously, 2 coeditors-in-chief have to keep in regular contact to ensure consistency.

JPN, like many similar journals, can publish only a minority of the manuscripts submitted. It is the journal's policy to reject without review and with minimal delay those manuscripts that will not receive the priority rating needed for publication even if external reviews are positive. The most common reason for rejection without external review is that the topic is not suitable for the journal. Unlike many other journals focusing on neuropsychopharmacology or biological psychiatry, JPN does not accept articles that are primarily of clinical interest. We aim to publish papers at the intersection of psychiatry and neuroscience. In its earlier days, the journal published a greater range of articles. However, with a large increase in the number of submissions, we can focus on the original objective of the journal, as indicated by its title.

The other main reason for rejecting manuscripts without review is the presence of methodological flaws, the most common of which is lack of statistical power. Sometimes the central purpose of a study is to demonstrate the equivalence of 2 groups, which is of course not the same as saying that the 2 groups are not statistically different. To demonstrate equivalence, it is important to determine the power of the study to show equivalence within a specified margin. On the other hand, in some cases a small sample size may lead to false-positive results. For example, there has been much discussion in the literature about why many studies reporting an association between a particular gene polymorphism and a symptom or disorder are not replicated.2,3 One reason is certainly that a small sample size is common in such studies, and JPN now routinely rejects such reports without external review unless the sample size is larger than that in most studies of this type. Association studies may be cited frequently, but JPN does not want to add further confusion to the literature in this area.

If an article is to be reviewed, the first task is to select reviewers. Those submitting manuscripts to JPN are required to suggest the names of 2 potential reviewers. Sometimes these suggestions are useful, but most are not. Suitable reviewers should have several qualities. First, they must have demonstrated that they are capable researchers. Those who have published fewer than 10 peer-reviewed papers are not suitable, although they are sometimes suggested. Second, they should not have published with any of the manuscript authors in the past few years. This may seem obvious, but a few authors do suggest collaborators as reviewers. Third, they should not suggest researchers who are unlikely to review. Authors will not know with any certaintly whether reviewers they propose are in fact willing to do so. However, a prolific and influential researcher who chairs a department and is involved in many important committees is very unlikely to review for JPN (Science or Nature perhaps!) and is probably not worth suggesting. Finally, a reviewer must have knowledge of the methodology used in the study. Suggesting that a psychiatrist who is an expert in the treatment of depression review a study on a rat model of depression is not helpful, nor is suggesting that an expert in the neuropsychological testing of patients with attention-deficit hyperactivity disorder review a functional magnetic resonance imaging study on the disorder.

Many manuscripts are submitted to more than 1 journal before they are accepted for publication. Each journal may have the manuscript reviewed by 2 people, so the total requirement for reviewers is several times the number of papers that are published. If scientists all reviewed twice as many manuscripts as they publish each year, the situation would be easy. However, many researchers seldom or never review manuscripts, increasing the burden of willing reviewers. Being an editor has greatly increased my respect for those selfless individuals who regularly produce informative reviews for little benefit other than the knowledge that they have contributed to the greater good (although knowing what the competition is doing or having advance knowledge of new research may also play a small role). Finding suitable reviewers is not a simple task.

Manuscript Central, the web-based reviewing system used by JPN, keeps a record of reviewers that includes when they last reviewed for JPN, their total number of reviews, the number of times they accepted and declined to review, their average response time to produce a review and the mean editor's rating of the quality of their reviews. This information is useful because it helps us avoid asking someone who has always declined to review and prevents us from making too many demands on willing and capable reviewers. One of the more time-consuming aspects of my job as an editor is searching databases and university websites for suitable reviewers and their contact information. Usually, an invitation to review will initially go out to 5 people with the aim of getting 2 good reviews. Some never accept or decline. Others may take up to a month to accept. Some accept but never get around to submitting their review. One of the arts of being an editor is knowing when to invite additional people to ensure that the review process is timely and how to avoid having too many eventually agree to review, which is a waste of a precious resource — reviewers' time — and may annoy reviewers. The JPN record for the number of reviewers invited to ensure 2 good reviews for a single manuscript stands at 16; fortunately, the number is usually much lower. Reviewers' unpredictability is the reason why I cannot answer our authors' most frequent question: When will I get a decision on my manuscript? From my experience and that of my colleagues when submitting manuscripts to other journals, I find JPN's time from submission to first response to be very competitive. However, comparisons are difficult. I have seen at least 1 case where a journal mentioned an average time from submission to first response that clearly included manuscripts rejected without review.

As soon as it appears from reviewers' comments that a manuscript may be accepted, the manuscript is sent to the journal's statistical reviewer. Serious statistical errors are common even in some high-profile journals,4 which is why JPN is fortunate to have on the editorial board someone trained in statistics who also has experience in applying statistics in psychiatric research (Lawrence Annable). Few papers pass statistical review without any change, and on occasion, it has been necessary to ask the authors for the printout from their statistics program to determine what they actually did. The nonstatistical reviewers seldom comment on the statistics, and when they do, their comments sometimes show their lack of understanding. This may explain why statistical errors are so common in the literature and reinforces the need for specialized statistical review.

Three decisions on a manuscript are possible once all the reviews are in. The first is rejection if the reviewers find important flaws in the study or consider it of minor importance. The second is to request the author to respond to the reviewers' comments and revise the manuscript and then to send the revised manuscript out to the original reviewers. Nearly all reviewers are willing to review a revised manuscript. This option is used when there are issues in the original manuscript that preclude a decision. For example, it may not be clear whether reported differences will remain statistically significant when the statistical tests are done properly, or there may be insufficient information given to determine whether the methodology was adequate. The third option is for the author to respond to the reviewers' comments by making minor revisions to the manuscript and for the editor to then assess whether the changes are satisfactory.

Decisions about manuscripts are not made according to the reviewers' majority vote. For example, I encountered a case in which 1 external reviewer gave a clearly articulated rationale for accepting a manuscript, whereas the other 2 suggested rejection. Of the 2 recommending rejection, the first thought the authors should have done a different study but did not clearly explain this opinion, and the second was similarly vague about the reasons for suggesting rejection. After carefully assessing all the opinions and looking at some related papers in the literature, I accepted the manuscript for publication. Both reviewers who recommended rejection sent me angry emails saying that I had wasted their time and that they would not review for JPN again. (I would not have asked them again.) In short, both authors and reviewers need to make their case with clear and rational arguments, and some authors do a much better job than others of arguing coherently against a point made by the reviewers. An emotional tone in either a review or a response makes it more difficult for me as editor to discern what is a valid difference of opinion and what is less-than-optimal science.

The most difficult task for an editor is to try to maintain consistent standards for acceptance across a wide range of disciplines. Obviously, there is an important element of subjectivity here. I have my biases that are probably different from those of other journals' editors. In general, I do not hold totally to my own perspective, or to majority views either, insofar as I can discern them. The success of any journal will ultimately depend in part on editors' decisions regarding manuscripts that are close to the cut-off. Whenever I reject a manuscript, I always try to keep in mind that my own 2 most cited papers were rejected by 3 journals before being published. Making mistakes as an editor is inevitable; the objective is to make as few as possible.

Authors of rejected manuscripts can appeal the decision. On appeal, a different member of the editorial board assesses the manuscript and the reviews. There have been few appeals so far, and they have occurred only when the rejection letter has not explained the decision sufficiently. This is why I try to explain my decisions to reject clearly if the reason is not absolutely clear from the reviewers' comments.

Clear communication is desirable but not always present in all aspects of research publishing. A recent editorial in Science mentioned that various sources, including a reader survey, suggested that “the language used in Reports and Research Articles is sufficiently technical and arcane that they are hard to understand even for those in related disciplines.”5 This is a problem for all research journals and certainly for JPN, which includes studies ranging from animal research and research on cells in culture to studies in patients. Once a study is complete, the single most important step that an author can take to maximize the possibility of publication and minimize the delay before publication is to write the manuscript clearly, with a minimum of jargon,6 and in a way that is accessible to any neuroscientist or psychiatrist. Obviously this is not easy, particularly for those whose first language is not English. Although JPN articles are copyedited for style and comprehension (not always the case in other journals), this usually involves only fine-tuning individual sentences. Many scientific articles tend to be dense, and sometimes, they are almost incomprehensible. I — and I am sure the reviewers — try to judge a manuscript by the science, not by the elegance of its English. However, it is much easier and more enjoyable to read a well-written manuscript. It would be surprising if good writing did not create some subtle bias. It is certainly my experience that reviewers are more likely to make mistakes in assessing manuscripts (such as asking for data to be included when it already is) when the writing is poor or the structure disorganized.

Authors can speed up the process and help create a good impression with the reviewers and the editor by following the guidelines below. If you do not, and the manuscript is to be published in JPN, you will have to make changes during the review process.

1. Describe the methodology in sufficient detail that another researcher could repeat the study.

2. Give full details on how the data were analyzed. Just saying that the data were analyzed by analysis of variance is not usually sufficient. What test was used to determine whether differences between 3 or more groups were significant? If multiple comparisons were done, what adjustment was made in the level of significance accepted? All details, such as values of test statistics and degrees of freedom, should be given, not just a simple p value. In a significant minority of manuscripts the degrees of freedom are not consistent with the sample sizes stated or the particular analysis carried out. This does not create a good impression. If an author does not make the effort to get the numbers right, what other aspects of the study were possibly carried out in a less-than-optimal way?

3. Specify whether the number after “mean ±” is standard deviation or standard error.

4. Make sure the numbers add up. For example, the sum of all the subgroups should equal the size of the full group. This type of oversight is surprisingly common.

5. Define technical terms that are specific to the methodology you use if there is any chance that some readers would not understand them.

6. When first used, define all abbreviations, except for ones that are generally known (such as DSM), and use abbreviations only if the term is repeated often.

7. Include a caption with all figures so that they can be understood without reference to the article's text.

8. Mention the name of the ethics committee or the animal care committee that approved the research. A study a few years ago found that 9% of the reports of clinical trials in high-profile journals made no mention of informed consent or ethics committee approval.7 Research not only has to be ethical, it also has to be seen to be ethical.

9. In research with human participants, mention that informed consent was obtained. If the research involved participants who might or might not have been able to give informed consent (e.g., patients with schizophrenia, mania or Alzheimer disease) mention whether the participants gave consent or whether someone consented for them. If they gave consent themselves, mention how it was determined that they were capable of giving fully informed consent.

10. Cite only necessary references. Space is a concern for all journals that are published on paper as well as online. The current JPN record is a manuscript that cited 16 references for a statement in a single sentence.

Just as editors expect manuscripts to be well written and logical, authors have the right to expect the same qualities in the editor's feedback. If there is something you do not understand in the comments on a manuscript, or if you feel that the editor or reviewers have made a mistake, please bring this to my attention. This can be done in a letter accompanying a revised manuscript, by telephone (1–514–398–7317) or by email. Whether or not a manuscript should be rejected has an important subjective element, and we may have to agree to disagree on that point, but if an author is left with the feeling that a manuscript was rejected on the basis of errors in fact or faulty logic, there has been a breakdown in communication. It is important to me as an editor that you as an author feel that you have been treated fairly because, in the long run, this is in the journal's best interests. This is why I always welcome feedback.