Peer Review in JAACAP Open: Promoting Integrity While Advancing Open Science

Abstract

A critical piece in the launch of JAACAP Open is the establishment of a high-quality and robust peer review process for incoming submissions. Indeed, peer review is the backbone of our scientific process. Here, we will discuss the importance of peer review, describe the process as we are expanding the JAACAP journal family, and explain why and how you can be involved in the peer review process.

A critical piece in the launch of JAACAP Open is the establishment of a high-quality and robust peer review process for incoming submissions. Indeed, peer review is the backbone of our scientific process. Here, we will discuss the importance of peer review, describe the process as we are expanding the JAACAP journal family, and explain why and how you can be involved in the peer review process.

What is peer review?

Peer review is the process by which one or more content or methods experts in a subject area examine a submission to the Journal. The purpose of peer review is to establish that the paper is high quality, important to the field, and free of bias.¹ In the case of research submissions, including systematic reviews and meta-analyses, there is the additional need to ensure that the methods are appropriate, that the results are valid and unbiased, and that the description of the research is adequate to allow replication and next studies. The conduct of peer review is critical to ensure the integrity of the work that we publish.

Why not just post to the internet?

With the development of multiple self-publishing options, including preprint servers (eg, bioRxiv, PsyArXiv are the most commonly used in our field), more research is being published on the internet without peer review. The risk of relying only on this method is that the inaccuracies in methods and interpretation often caught by peer review may not be identified. This was especially evident during the rush to disseminate new information during the COVID-19 pandemic.² For open access journals such as JAACAP Open, the goal is to provide rapid public access to new information online, with the additional benefits provided by peer review. Peer review enhances rigor by improving study design and methodology, increasing transparency in data reporting and analysis, and strengthening the reliability of reported research. In addition, having work reviewed by peers may increase innovation by pushing authors to balance innovations with established theories in the literature via the constructive feedback offered by reviewers.

Are there benefits to the peer reviewers?

There are many benefits of peer reviewing to the peer reviewers as well. There is, of course, the overall opportunity to contribute to the scientific process and become a more critical reader of the literature. Those who review regularly for a journal are often invited to join the masthead or editorial leadership, providing a unique opportunity to influence the publication of articles in our field. There is also the prospect of improving one’s own writing and research by reading and commenting on cutting-edge findings and research methods. For those in academic settings, there is the added advantage of meeting service expectations for tenure and promotion and to build and reinforce an international reputation as a field expert. There is the opportunity to interact with editors at the Journal, establishing connections and collaborations.

How does the process work?

A graphical description of the process is shown in Figure 1. A submitted manuscript first undergoes administrative (ie, assessment whether the manuscript fits the scope of the journal and minimum formatting expectations) and scientific (ie, assessment whether the manuscript meets the quality and methodologies that are expected in the journal) review by the editor. The editor decides whether to send the manuscript to peer review and, if so, decides to manage the manuscript themselves or assigns the paper to a handling editor (the associate editor or one of the deputy editors). If there is a conflict of interest with the manuscript (eg, if the manuscript is from a member of the senior editorial team), an external editor is requested. The editor assigned to manage that manuscript is considered the action editor (AE). The AE now has the responsibility to review the manuscript and again decide whether it should be sent to peer review. If they decide to send the manuscript to peer review, the AE requests peer review from one or more reviewers determined to be experts in the content area of the manuscript. Reviewers are invited to review. If they agree, the manuscript is sent to them. Although there are many different models of peer review, JAACAP Open is a double-anonymized journal, meaning that the identities of the reviewer and author are hidden from each other, and the reviewer receives a deidentified copy of the manuscript. This is to allow more objectivity and constructive feedback. If a bias is evident to the reviewer or to the editor for a manuscript, they are expected to recuse themselves from the handling of that submission. Peer review for JAACAP Open is voluntary and unpaid. While this approach has its critics,³ the expectation is that we are a community that supports each other without the necessity of compensation.a

Figure 1.

The Peer Review Process at JAACAP Open

Once they receive the manuscript, the reviewer is tasked with reading and reporting on the submission. Despite double-anonymization, there can be times when the reviewer identifies the origin or authors of the manuscript and may be in conflict. They are expected to identify these conflicts of interest or any other biases early in the process so that the manuscript can be reassigned. They are also expected to keep the submission confidential.⁴ In reviewing the submission, reviewers are asked to be professional, pleasant, and constructive. They are asked to be thorough and specific in their critiques. The structure of a good review includes a one-paragraph summary that compiles what the reviewer has learned about the paper and gives an evaluation of the overall strengths and weaknesses of the submission. Following this, the review should contain specific comments and suggestions. For research submissions, the evaluation of the introduction should determine if the context of the work is placed correctly in terms of the current literature and theories and should determine if the hypotheses are stated accurately and without bias. Review of the methods should contain an assessment of the sample, the measures, and the statistics used. Not every reviewer is expected to know every statistical technique (the Journal also has specific Methods and Statistics reviewers). The “Results” section is reviewed for an accurate description of the sample (importantly for the presence of a description of the race, ethnicity, and sex distribution of the sample), the reporting of critical findings, the accuracy of the tables and figures, and whether there is any selective reporting of findings. The “Discussion” is then reviewed for whether the findings are appropriately placed into context without too much “spin”^5,6 and whether limitations are identified appropriately. These findings are placed into a written review regarding the importance and impact of the manuscript that allows the AE to make a decision on the manuscript. The written report is submitted to the AE, who determines whether to accept, reject, or ask for revisions to the manuscript. One or two rounds of revision and re-review may then occur. (For more information about the review process, see https://www.elsevier.com/reviewers/what-is-peer-review.)

The review and re-review process is iterative between authors, AEs, and reviewers. The AE is charged with assessing the quality of the manuscript as well as any biases in the reviews and whether the critiques will improve the product. Balance needs to be struck between scrutiny, collegiality, and the needs of the readers. The AE then reports their opinion to the Editor of JAACAP Open to make a final decision on the manuscript. In the end, this is the critical role of the Editor—to ensure that the final product is the highest quality for the reader and for the field. The senior editorial team (Editor, Associate Editor, Deputy Editors, and Managing Editor) also has an obligation to address ethical dilemmas and confidentiality issues and to ensure fairness, diversity of opinion, and inclusivity in the peer review process.

Are you ready to contribute to peer review?

In short, yes. The peer review process at JAACAP Open is designed to be fair and objective and consequently requires a diversity of thoughts and opinions.⁷ Moreover, it does not require expertise in every area of a manuscript from every reviewer. The job of the editorial team is to ensure that there is adequate coverage of expertise in the review of each manuscript, and we can use more reviewers. If you are interested in joining the peer review team at JAACAP Open, we invite you to visit the JAACAP Journal Family Editorial Opportunities portal⁸ to express interest and be considered for future opportunities to peer review. If you do not feel quite ready, please still reach out and we can help get you the training that you need, including our Peer Review Workshops at the AACAP Annual Meetings, our guide for reviewers,⁷ and other self-guided educational opportunities on our website.⁸ We look forward to working with our reviewer teams to support the highest rigor and quality in papers that we publish at JAACAP Open.