Peer reviewing an original research paper

In our earlier editorials on research and medical writing, we have discussed the importance of research; we have given a roadmap for drafting an original research article and have also provided suggestions on how to publish research papers successfully.[1,2,3] In this editorial, we discuss the process of peer review and method for writing good reviews for original research papers. The authors, reviewers, and editors form the vital links in the process of publication and scientific communication and hence they need to understand their roles and “shared” responsibilities in this process.[4,5] The authors need to ensure that they adhere to good science and ethics while the reviewers need to be meticulous, unbiased, sincere, supportive, and constructive while giving their reviews.[4,5] The editors need to be unbiased and disciplined and follow best practices while handling the manuscripts received by the journal.[4]

The editors do largely depend on the opinions given by the reviewers of a manuscript while making critical decisions, though they are not obligated to do so.[4,6,7] Peer review is primarily the assessment/evaluation of research by an expert in a particular field.[4] It is a critical component of the publication process and helps to maintain a high quality of published research.[4,6,7] With the explosion in the number of scientific biomedical journals and the large numbers of manuscripts submitted for publication, identifying appropriate reviewers, and retaining their services has become increasingly difficult for the journals. Prestigious indices such as the Index Medicus/PubMed and Scopus necessitate that the journals indexed with them follow a proper peer review process.[4] It is better to use the term “assessor” or “reviewer” instead of the older term “referee” as the final decision regarding a manuscript has to be taken by the editor and not the reviewer.[7]

The peer review process

This has been given elaborately in our earlier editorial on success in publishing a paper.[3] Once a paper is submitted on the website of a journal, the editor/subeditor checks the paper for completeness and adherence to their journal style. Permission from the Institutional Ethics Committee and compliance with appropriate informed consent/assent are checked. The paper is then circulated through some of the editorial board members for an “internal review” regarding the relevance, ethics, and methodology. The editorial board members submit their opinion (with relevant explanation) on whether the manuscript is worth processing for a formal peer review. The manuscript may be rejected at this stage if it is not suitable for the readership of the particular journal or if it has glaring ethical/methodological faults.[5] Only after passing the “internal review” process successfully, the manuscript is sent for an “external review” to the peer reviewers.

The editor/subeditor usually invites (usually by email) three to five experts in the field of the research paper to review the manuscript.[4,7,5] More reviewers get invited if some of these initial reviewers decline to review. On acceptance of the assignment, the reviewers are usually given about 4 weeks to perform the peer review. At times, however, the editor may ask for a rapid/fast review.[7] The manuscript management system can send automatic reminders to the reviewers.[7] The reviewers upload their comments to the authors and editors separately through the manuscript management system, with a confidential recommendation on acceptance/rejection/revision of the manuscript (this recommendation is sent to the editor only). The authors and reviewers are usually blinded to each other. Based on the reviews received and his/her own assessment of the manuscript, the editor/subeditor then takes a decision whether to reject the manuscript or invite a revision (minor/major revision) from the authors and the authors are informed accordingly. If a revision is invited, the authors are asked to give point-by-point reply to each of the reviewer's comments and to state the exact location in the (revised) manuscript text where the change has been incorporated (by underlining/highlighting the relevant changes or by colouring the revised text font).

Depending on the nature of the comments offered by the reviewers, the editor may take a decision whether to send the revised manuscript back to the original reviewers for their opinion (for deciding the adequacy of the revisions made by the authors). Most often the editors do send the revised manuscript back to the original reviewers for a rereview. At this stage, the reviewers can view the comments offered by the other reviewers, so that they can understand why particular changes have been made in the manuscript, which they did not advise for (and this helps as a learning tool for the reviewers as well).[6] The reviewers are requested to give their opinion on the revised manuscript in a similar fashion as the first review with comments to the authors and the editors separately. This process may continue until the manuscript is either rejected or accepted (with the changes asked by the reviewers and editor/s).

The editor/s respects the recommendations given by most the reviewers but make the final decision on their own (this decision may, at times, be contradictory to reviewer recommendations). In case of a tie on recommendations from the reviewers, the editor/s or a member of the editorial board may take a decision after a re-evaluation of all the reviewer opinions or an additional reviewer may be roped in for help in making a final decision.[5] The final decision that is made (acceptance/rejection) is usually conveyed to all the concerned reviewers.[6] Occasionally the editor/s may invite additional reviewers in case of conflicting reviewer opinions/recommendations. Editor/s consult their statistical consulting editor/statistical reviewer for papers with unfamiliar/difficult to understand (“heavy”) statistics.[5,7,8,9] The editor may also invite one of the reviewers or subject experts to write an editorial commentary on the research paper which has thus been accepted for publication.

If the authors are unhappy with the editor's decision, they may represent their case back to the editor and if they are not satisfied with the same, they may approach the ombudsman/ombudsperson (a “neutral” person) appointed by the journal.[4] The ruling given by the ombudsman is honoured by the editor as well as the authors.[4] Some manuscript management systems prompt the authors to suggest preferred or nonpreferred reviewers while uploading their manuscript and authors can avail this facility.[4,5,7,8,10] However, it is not mandatory for the editor to necessarily choose the reviewers suggested by the authors.[4,5,8]

The “blinded” versus the “open” peer review

Most journals follow a “single-blinded” (authors are blinded to the names of reviewers) or “double-blinded” peer review process (where the authors and reviewers are unaware of each other's identity).[4,5,7,10,11,12,13] The advantages of this “double-blinded” system include maintenance of confidentiality, likelihood of the reviewers giving an honest opinion/critique, protection of authors from their ideas getting stolen/plagiarized, and protection of junior reviewers from possible retribution from senior authors (since the identity of the junior reviewer is not known to the senior author/s).[4,7,11,12] The disadvantages are that the reviewers may provide a very rude/harsh or extremely critical review (lesser accountability arising out of the reviewer anonymity) and a dishonest reviewer may use the process for self-promotion or some other ulterior motive (like plagiarism of the author's idea/hypothesis).[4,7,11]

The “open” peer review system is where the authors and reviewers know each other's identity.[4,7,10,11,12,13] The advantage of the “open” peer review system is better accountability (and hence civility and courteousness) of the reviewers.[7,10,11] However, the disadvantages are that the reviewers may provide either noncritical reviews or decline to review papers of authors known to them.[4,7,10,11,12] At times, even in a blinded review process, the reviewers can guess the names of the authors/author-group, especially if they are working in a particular (similar) restricted field of science.[4,7,11,14] Some journals may ask the reviewers to sign their reviews which are posted in the public domain for improving transparency.[7,12]

Who can be a reviewer

A reviewer must have published a reasonable number (at least 5) of original research papers in the past so that they are well aware of medical writing and the peer review process. The editor invites a scholar/academician who has published on the topic similar to the research paper (in hand) or an expert who is a well-known authority on the particular subject or one who has successfully reviewed for the journal in the past.[5] While registering as a reviewer on the manuscript management website, the reviewer is asked to give his/her area/s of expertise (“keywords”); this also helps the editor in choosing appropriate reviewers.[7] Most journals have a database of the potential reviewers for their own journal/consortium and the editor can select reviewers using “keywords”.[5,7,8] An interested researcher may send his/her curriculum vitae to particular journal/s and request for empanelment as a potential reviewer.[4] Being a reviewer is considered to be an honour, privilege, and duty of researchers/academicians and one should (usually) not refuse to review a paper if they have the expertise to do so and ensure that they can spare their time to complete the review.[4,7]

Reviewing is an unpaid/voluntary work used as a measure of academic esteem by universities and usually not rewarded with any money.[7,8,15] Most journals publish the names of those who have reviewed for their journal once a year as a tribute to the reviewer's services and may reward the reviewer by giving complimentary access to their journal/group of journals for a limited period of time.[7,11,15] On average, reviewers spend up to 5 h (median time used) to 9 h (mean time used) for each review and review regularly for about three to four journals.[7,11]

Training of reviewers

Most reviewers learn the review process by trial and error (i.e., self-taught).[5,8,15] Though no particular training is necessarily available for one to become a reviewer, training oneself using tutorials available on the manuscript management websites or journals websites or by attending workshops/editorial fellowships and reading literature/books on peer review, does help.[4,5,7,10,12,13,15,16] Some senior academicians may mentor their junior colleagues as well.[5,8,15]

Attributes/characteristics of a good reviewer

A good reviewer provides a useful and timely review of the manuscript. Table 1 gives the attributes/characteristics of a good reviewer.[2,3,7,10,14,15]

Table 1.

Characteristics of a good reviewer[2,7,10,14,15]

Characteristics of a good reviewer
A good reviewer should possess the following characteristics:
Should have published original research articles himself/herself.
Needs to be knowledgeable in his/her own field/area of expertise, needs to keep updated with the current literature, needs to possess critical thinking ability and should have expertise in the field in which the paper has been submitted.
Needs to have a sound understanding of basic medical statistics and epidemiology.
Should have reviewed for other journals (preferable).
Provides a constructive, objective, fair, thorough, honest, and polite review.
Has time and inclination/aptitude to review papers and is quick to respond to review requests from the editors.
Is ready to work as a reviewer without expecting any rewards/returns (i.e., work voluntarily and without any honorarium i.e., altruistic nature) in the best interest of advancing science.
Should be able to adapt to the changing landscape in publishing and keep updated with the latest trends in the publication process.
Can identify own/personal biases and minimize them.
Acts as an “ally” or a “friend” or an “advocate” of the author and help the authors in improvising the manuscript (irrespective of the final outcome of acceptance or rejection by the journal).
Has good knowledge of English language and grammar.
Has good “academic ethics”.
Comes from a top academic institution with “publishing culture”.
Can maintain a balance between quality control and encouragement of innovation.

Responsibilities of a peer reviewer

Reviewing a paper is an academic privilege.[7] With authority, there exist great responsibilities that the reviewers should be ready for. Some of them are enlisted in Table 2.[4,5,6,7,8,9,10,11,12,14,15,17,18]

Table 2.

Responsibilities of peer reviewer/s[4,12,14,15,17,18]

Responsibilities of peer reviewer
A peer reviewer holds the following responsibilities:
Should maintain utmost confidentiality as regards their assignments and should not discuss the same publicly or with colleagues.
Should not delay the submission of their review/report purposefully. If the reviewer does not have the necessary time to review, he/she should immediately write back to the editor (when initially invited to review), so that the editor can choose another reviewer. If he/she has already accepted to review and is getting delayed in the submission of the review, the same should be brought to the notice of the editor. He/she can also suggest alternate reviewers to the editor/s.
Should excuse/recuse themselves from the peer review if they have actual/perceived conflict of interest or bias.
Should not plagiarize the ideas of the authors.
Should not use the contents of the unpublished manuscripts sent to them for furthering their own interests.
Give adequate time for reviewing the manuscript and read relevant contemporary literature (if necessary).
Should not accept review requests on topics outside their expertise.
Should use courteous language.
Should never correspond with the authors directly. All reviews/comments should be submitted to the editor.
Should give a recommendation regarding rejection, revision, or acceptance only if asked to do so by the editor.
Should not pass the manuscript to juniors or colleagues for helping to write the review. If they cannot review, they should state accordingly to the editor. If they need help or want to mentor juniors, they should obtain permission from the editor for the same and ensure that the junior reviewer is given the necessary credit for helping in the review.
Should contact and inform the editor immediately if scientific fraud, author misconduct, or plagiarism is suspected.
Should not retain the manuscript for their personal use and destroy the paper and electronic copies after submitting their reviews.
Should not allow personal biases to affect the content of their review.
Should act as “advocate” of the author (i.e., help the authors in improvising their manuscript and explain their research idea to the reader) and the journal (help to maintain the high-quality of the manuscripts accepted by the journal) as well.
Should treat all manuscripts in a way he/she would want his/her manuscript to be treated.
Should not give vague or generic criticisms but should be more clear, objective, and fair.

Advantages and disadvantages of the peer review system

Peer review ensures that the research published in a journal has a good quality. This is probably the only method of identifying publishable research from numerous manuscripts received by a journal. The advantages and disadvantages of the peer review system are given in Table 3[4,5,7,10,11,12,14,15,17,19,20] and Table 4,[4,5,7,10,12,13,14,16,20,21,22] respectively. Recently, there have been peer reviewer frauds/scams and rigged peer reviews i.e., authors creating fake email addresses and suggesting the same to editors as possible reviewers and reviewing their own manuscripts; fabricated reviews by agencies that help authors to write manuscripts and then selling them favourable reviews; and cyber fraud by hacking/manipulating the peer review computerized systems by unscrupulous authors posing as expert reviewers and reviewing their own manuscripts. This has led to the retraction of such papers by various journals.[12,14,16,21]

Table 3.

Advantages of the peer review system[4,5,7,10,11,12,14,15,17,19,20]

Advantages of the peer review system
Can detect major faults in the originality of the article, the science, the methodology, and the ethics of the paper.
Improves the quality of the manuscript and confers “added value” to the manuscript.
Detects whether the manuscript is as per the requirements of the journal.
Facilitates the provision of important and credible science to the readers.
Gives respectability and credibility to the journals which have a systematic peer review system in place.
Gives the scientific community an opportunity to recognize and support innovative research.
Minimizes duplicate and redundant publication.

Table 4.

Disadvantages of the peer review system[4,5,7,10,12,13,14,16,20–22]

Disadvantages of the peer review system
The output depends on the efforts taken by the reviewer and the quality of reviews may vary accordingly.
Potential for reviewer bias exists. Personal biases may lead to the reviewer having a closed mindset and suppress new ideas.
May still miss major flaws in the manuscript.
It is expensive for the journal.
Publication may be delayed due to the time consumed in conducting the peer review.
May not detect a conflict of interest by the authors.
Editors may assume a “high-handed” or a “hands-off” attitude.
It may be difficult for the editor to find reviewers as the existing ones are already overloaded and there is a dearth of good reviewers.
Scientific frauds may (still) go undetected despite the peer review process.

Writing the actual review

The reviewer should be acquainted with the instructions to authors and the readership of the journal that they are reviewing for. The reviewer is expected to give a separate set of comments to the authors and to the editor. The comments to the authors relate to the science and draft of the manuscript; while the comments to the editor are regarding the overall manuscript evaluation and recommendation on acceptance or otherwise (these “confidential” comments to the editor cannot be viewed by the authors).[5] The reviewer is expected to help the editor in making a decision on the suitability of the manuscript for publication and how a manuscript can be improved upon.[7] Unacceptable flaws related to ethics or methodology need to be identified by the reviewer (this is, at times, already done by the editor/editorial board members in their “internal review”).[7,15] The reviewer, hence, even if an expert at a given topic, needs to read more literature about the topic to decide about the originality and quality of the paper.[7,15] Occasionally, the reviewer may unearth misconducts such as duplicate publication or salami (splitting research to create multiple papers) publication or plagiarism in the process of reviewing the (current) literature.[7,15]

Comments to authors: It is advisable to start with a summary stating the essential crux of the paper.[7,8] This assures the editors and the authors that the paper has been read completely and understood by the reviewer. Then either list the concerns as “major” and “minor” concerns or proceed section-wise (that is from the title, abstract, introduction, etc. up to the references section) to give specific comments.[7,8,17] Table 5 gives a format of a checklist which will help the reviewers in giving a complete review for a research article.[2,3,5,8,9,11,15,17,18,23] Such a checklist also helps the reviewers and reminds them of the various items/aspects that they need to evaluate in a research article. Standardized checklists like the CONSORT (consolidated standards of reporting trials) and STROBE (strengthening the reporting of observational studies in epidemiology), which are available at www.equator-network.org and used by authors (to draft their manuscript), are usually mentioned in their instructions to authors by the journals and can also be used for assisting the reviewers.[2,7,8,9,13] Similar checklists/questionnaires are used by the journals as well, and the reviewers are asked to rate the manuscript based on the same.[5] It is a good idea to number the comments so that the authors can reply point-wise (should a revision be invited from the authors). The primary work of the reviewer is to assess the science and ethics of the manuscript. It is not mandatory to comment on the grammar or syntax as this is the work of the copyright editor/assistant; however, guidance on language is welcomed by the editors. Though there are no recommendations on the length of the reviewer's report, about 1.5 to 2 single-spaced pages are usually enough.[8] The “gatekeepers” of scientific communication (editors and reviewers) need to review the scientific/discipline-based content (the “screening” function) of a paper as well as look at improving the paper as a tool for written communication (the “improving” function).[19] Some reviewers may prefer to mark their comments on a pdf or word version of the submitted manuscript and upload the same for the authors viewing and revision.[5]

Table 5.

Checklist for giving a complete review for a research article[2,3,5,8,9,11,15,17,18,23]

Section	Items to be checked
Title	Whether appropriate, clear, concise and precise. All essential components (population, intervention, control, and outcome i.e., PICO) are mentioned (may vary with the type of the study).Related to the topic/content of the paper and does not (usually) reveal the result.
Abstract	Concise and follows guidelines to authors stated by the journal. Usually structured- Background/introduction, aims, methods, results, conclusions (or similar subsections). Whether it is independent, complete and stand-alone (i.e., give a fair idea of the contents of the paper without reading the complete manuscript). Reflects the content of the article accurately (especially after a revision). Is it consistent with the content of the main text? Word limit stated by the journal is adhered to (usually about 250 words). Keywords from MeSH terms (Pubmed website) are correctly mentioned at the end of the abstract
Introduction	Gives the rationale in brief. Gives adequate background information and context. Gives only the relevant literature and few (four to eight) appropriate references. Is the research question important? Whether the objectives and purpose of the study are clearly stated.
Materials and Methods	Whether ethical clearance and informed consent/assent have been mentioned. Appropriate sampling procedures have been used. Whether the study participant selection (and description ), study design, randomization, blinding, data collection procedures and instruments, precise details of interventions, study treatments/procedures, study endpoints, and primary/secondary outcomes have been stated. Is the sample size adequate to provide the necessary statistical power to the study (minimize type II error)? Statistical test(s) are described and appropriate. Are trade names and symbols used properly? If diagnostic kits or statistical packages or questionnaires are used- whether details like owner/producer/trademark-/patent-/copyright-holder stated, and whether the city, year and permission to use is stated.
Results	Whether well presented, clear, precise, and concise. All points raised in the methods are answered and results for all endpoints are stated in a logical sequence. Whether actual P values and 95% confidence intervals (CI) are reported. Whether the tables, figures, graphs, charts, and photographs are used appropriately and improve the readability of the manuscript. Have appropriate legends been given for these? Whether some tables can be simplified, condensed, or omitted. Whether appropriate units are used to describe the data. Whether the results are reported in relation to the specified aims and objectives. Whether the statistical analysis is adequate and appropriate. If supplementary files/materials are supplied, these also need to be reviewed by the reviewer.
Discussion	Whether it summarizes findings and explains the meaning of the main result. Emphasizes the new and important aspects of the study. Is in relation to the objectives/hypothesis. Answers “how” and “why” of the manuscript. Whether the literature search is adequate, and comparison is appropriately done. Whether the interpretation of results has been given. Is the result clinically important? Whether the main conclusion/s are explained and supported by the results. Whether it explores plausible explanation/s for conflicting results. Whether generalizability of results has been stated. Practical implications, strengths of the study as well as the limitations and biases have been discussed. Directions for future research have been stated. Whether the conclusion is rational and take-home/key message has been given.
References	Vancouver style or as per journal instructions/requirement. Whether most of the references are recent (from the past 5-10 years). Whether the number of references is within the maximum limit prescribed by the journal. Whether the references are cited correctly in the text.
Miscellaneous	Whether the paper contributes something new to the existing knowledge (novelty). Whether the paper gives clear and concise information. Whether the paper follows a logical sequence and avoids repetitions. The writing style is appropriate as per the journal’s instructions to authors. Correct use of English grammar and spellings.
Originality	To be rated (Good/Fair or Adequate/Poor).
Clinical relevance	To be rated (Good/Fair or Adequate/Poor).
Appropriateness for the readership of the journal	To be rated (Good/Fair or Adequate/Poor).

Comments to the Editors: The strengths and weaknesses of the paper should be clearly identified by the reviewer in their confidential comments to the editors.[5,8,15,17] The recommendation (if) given by the reviewer (reject, major revision, minor revision, or accept) should be supported by their main review. The reviewers may suggest an appropriate journal section for which the manuscript can be assigned to. The reviewers may also ask for changes in the format/type of the manuscript (e.g., an original article to be converted to a brief original article/report or a research letter).[8] These recommendations are useful to the editors but not necessarily binding on them.

Recent concepts

Some journals have started using “postpublication” system, wherein the reviewed and the accepted manuscript is posted on the website in advance of the paper publication; readers comment on the same and such (reader's) comments can be actually published with the manuscript itself.[5,7,10,11] Many open access journals have started following the “light touch peer review” wherein the review is focused on the methodology.[7] Some journals publish the peer reviewer comments or the entire “prepublication history” (submitted versions, reviewer's reports, and author responses) and the timelines online (as a weblink) with the journal article.[7,12] There are journals where a reviewed manuscript draft is posted electronically for critique by the academic community and after a defined period, the modified draft paper is accepted for publication.[7,12] In journal review networks/peer review consortia, the reviews are transferred from one journal to another after rejection, this saves on reviewer/editor time and reduces duplication of work.[5,10,13]“Publons” (https://publons.com/about/home/) is a website which builds public reviewer profiles for participating reviewers. The “Publons” profile shows a researcher's verified review history across all journals, gives various reviewer metrics, and gives credits for work as a reviewer/editor. Useful guidance on ethics of publication and peer review is available on the COPE (Committee on Publication Ethics) website (https://publicationethics.org/resources/guidelines-new/cope-ethical-guidelines-peer-reviewers) and authors, reviewers, and editors need to be acquainted with these.

Concluding remarks

It is an honour and privilege to review papers and reviewers need to do it with a sense of responsibility, integrity, and altruism.[7,15] The reviewer needs to be polite and systematic while performing the review. Proper reviewer conduct makes the peer review process valuable and the journal becomes more trustworthy.[15] Peer review has thus been called the “heart and soul of scientific publishing”.[15] A systematic well-organized review is appreciated by the editors/authors and facilitates manuscript revision.[17] The entire peer review process is based on mutual trust between the authors and editors and also between the editors and reviewers.[15] This editorial has given an account of the peer review process and guidance for the reviewer to write good (and practically useful) reviews. It is hoped that this editorial will help the reviewers in their career in reviewing as well as assist the authors in medical writing.