Fate of Manuscripts Rejected by a Specialty Psychiatry Journal: A Retrospective Cohort Study

Abstract

Background:

Little is known about the publication outcomes of submissions rejected by specialty psychiatry journals. We aimed to investigate the publication fate of original research manuscripts previously rejected by the Indian Journal of Psychological Medicine (IJPM).

Methods:

A random sampling of manuscripts was drawn from all submissions rejected between January 1, 2018, and December 31, 2019. Using the titles of these papers and the author names, a systematic search of electronic databases was carried out to examine if these manuscripts have been published elsewhere or not. We extracted data on a range of scientific and nonscientific parameters from the journal’s manuscript management portal for every rejected manuscript. Multivariable analysis was used to detect factors associated with eventual publication.

Results:

Out of 302 manuscripts analyzed, 139 (46.0%) were published elsewhere; of these, only 18 articles (13.0%) were published in a journal with higher standing than IJPM. Manuscripts of foreign origin (odds ratio [OR] 1.77, 95% confidence interval [CI] = 1.06–2.97) and rejection following peer review or editorial re-review (OR 2.41, 95% CI = 1.22–4.74) were significantly associated with publication.

Conclusion:

Nearly half of the papers rejected by IJPM were eventually published in other journals, though such papers are more often published in journals with lower standing. Manuscripts rejected following peer review were more likely to reach full publication status compared to those which were desk rejected.

Key Message:

• Less than half of the manuscripts rejected by the journal were published elsewhere.

• Rejected submissions were more often published in journals with a lower CiteScore.

• Manuscripts originating outside India and those rejected following peer review (as opposed to desk reject) were more likely to be eventually published.

Academic publishing is now a necessity for career advancement and securing tenured faculty positions. ¹ An unintended consequence of this development is that many journal editors are grappling with a problem of plenty; they are receiving many more submissions than they can eventually publish. This results in restrictive editorial policies for evaluating submissions. In an earlier editorial in the Indian Journal of Psychological Medicine (IJPM), we had written about raising the bar for initial evaluation of manuscripts submitted to the journal and had spelt out the kind of submissions that are likely to be desk rejected. ²

At the IJPM, the editorial team performs an initial screening of all submissions. Those which are out of scope, lack sufficient novelty, or have serious methodological flaws are usually desk rejected. The remaining manuscripts are then sent out for double-blind peer review. Following external peer review, all articles recommended for acceptance are moved for two further rounds of editorial re-review. The first round is a scientific review performed by two senior members of the editorial team with expertise in research methodology; the second round focuses on language and stylistic editing of the manuscript and is performed by the editor-in-chief. Only after the conclusion of these sequential processes does the editor-in-chief make a final decision (accept/reject) on the manuscript. A detailed description of the journal’s peer review process is available elsewhere. ³

Although well-intentioned, a potential risk with restrictive manuscript selection policies is that journals may miss some important papers that may end up being published in higher ranked journals ⁴ or, worse still, not be published at all. Admittedly, an eventual publication does not always imply quality, as shown in a classic analysis ⁵ ; nevertheless, good journals must continuously reflect on their editorial policies and tweak them, if necessary, to avoid “big misses.” One way of achieving this and indirectly vindicating editorial selection policies is to track the fate of rejected manuscripts periodically.

To our knowledge, no such analysis has been done in psychiatry journals despite its obvious implications for journal editors, peer reviewers, as well as authors; however, this has precedents in other broad ⁶ and narrow medical specialty journals.^{7, 8} To fill this gap in the literature, we carried out the present study with the primary objective of assessing the proportion of manuscripts rejected by IJPM that was subsequently published elsewhere. Additionally, we also examined the quality of the journals in which the rejected manuscripts are eventually published (to detect “big misses”).

Methods

Study Inclusion Criteria

This was a retrospective cohort study of rejected manuscripts which was carried out in November 2020. We included a random sampling drawn from all manuscripts rejected by IJPM between January 1, 2018, and December 31, 2019 (n = 669). There are three possible types of manuscript rejection at the journal: desk rejection, post-peer review rejection, and post-editorial re-review rejection. Desk rejection constituted two-thirds of all rejections at the journal during this period (n = 446, 66.7%), followed by post peer review rejection (n = 170, 25.4%) and post editorial-re-review rejection (n = 53, 7.9%). ³ Hence, to enhance the final sample’s representativeness, we randomly selected 20% of desk-rejected manuscripts and 50% of those rejected following peer review; all the manuscripts rejected following editorial re-review were included. Random sample was selected using the select cases dialogue box in SPSS statistics for Windows version 20.0 (IBM Corp., Armonk, NY) which allows sampling based on approximate percentage or exact number of cases. We selected the former option and because sampling is done without replacement, the same manuscript cannot be included more than once.

Data Abstraction and Coding

Using the titles of the rejected manuscripts and author names, we searched PubMed and Google Scholar to locate manuscripts published elsewhere; if nothing was found, a further search was done using keywords from the title. A psychiatrist researcher carried out the initial search using this sequential strategy; those articles that could not be located were independently searched by a second investigator using a combination of keywords from the titles and the names of the first author or the corresponding author. To ensure concurrence, for every located article, its abstract/full text was retrieved and compared with the original manuscript files submitted to IJPM.

For articles that were published in a journal, we abstracted the following information about the journal: name, presence in Kscien’s list/Beall’s list of predatory journals defined as “entities that prioritize self-interest at the expense of scholarship and are characterized by false or misleading information, deviation from best editorial and publication practices, a lack of transparency, and/or the use of aggressive and indiscriminate solicitation practices” ⁹ (yes/no), whether indexed in MEDLINE (yes/no), whether indexed in Scopus (yes/no), Scopus percentile score, impact factor as per journal citation reports 2020, CiteScore, journal quartile from Scopus (i.e., Q1 0–25, Q2 26–50, Q3 51–75, and Q4 76–100), and whether the CiteScore is higher than IJPM (yes/no), and whether it is a journal originating within India or outside India (yes/no).

Besides, the following details of the manuscript were abstracted from the journal’s erstwhile manuscript management portal (www.journalonweb.com): year of rejection, manuscript identification number, type of manuscript, type of first author (psychiatrist or non-psychiatrist), number of authors, whether the work originated from India or abroad (yes/no), whether the work has been previously presented elsewhere (yes/no), reasons for rejection, and type of rejection (desk-rejection vs. post peer review or post editorial-re-review rejection). Reasons for manuscript rejection were drawn from previously published data and the extracted reasons were grouped as “fatal” and “nonfatal” using the methods described therein. ³ Fatal flaws represent errors involving aspects central to the study that cannot be rectified once completed; examples include errors in study design or choice of study tools. ¹⁰ In contrast, nonfatal study flaws such as applying inappropriate statistical tests or inadequate comparisons with available studies represent errors that can be addressed at the review stage.

From the journals to which the rejected manuscripts eventually found their way, we prepared a list of top 10 journals with higher standing than IJPM. Because IJPM has a CiteScore of 1.6 ¹¹ but no impact factor as yet, we classified those journals with a CiteScore of more than 1.6 as having a higher standing than IJPM. Such metrics have been used previously in similar research. ⁴

A single author performed the data abstraction and coding; queries that arose during the coding process were clarified by mutual discussion till consensus.

Statistical Analysis

Data analysis was done using SPSS statistics for Windows version 20.0 (IBM Corp., Armonk, NY). We used descriptive statistics (frequencies with percentages) to depict abstracted characteristics of the rejected manuscripts and characteristics of the journals in which they were eventually published. We examined the association between reasons for rejection and publication in a journal with a higher CiteScore than IJPM using a chi-square test.

Multivariable analysis, using logistic regression, was used to quantify associations between publication status (published vs. not published) and covariates of interest (reasons for rejection, type of rejection, type of submission [original article vs. other types], type of the first author, number of authors, and origin of the manuscript). These covariates were decided a priori by consensus among the investigators. The covariates were first entered individually in logistic regression to generate univariable OR and later together to generate the multivariable OR. Associations were estimated using odds ratios (OR) and 95% confidence intervals (CI). A two-tailed P-value less than 0.05 was considered significant for all comparisons.

Ethical Clearance

As the work involved analysis of secondary data, no formal institutional ethics committee clearance was obtained.

Results

Baseline Characteristics of Included Manuscripts

A total of 302 rejected manuscripts were included for analysis (Table 1). The majority were submitted to the journal in 2018 (n = 161, 53.3%) and the others (n = 141, 46.7%) in 2019. Original articles constituted more than two-thirds of the sample (n = 205, 67.9%). More than three-fourths of the manuscripts were rejected because of the presence of fatal flaws (n = 230, 76.2%), and the maximum fraction comprised of desk-rejected manuscripts (n = 134, 44.4%). We were able to trace almost half of the rejected manuscripts (n = 139, 46%) online; of these, 138 (99.3%) were published in journals, and one was subsequently published in a book (Figure 1). Of the articles that could be traced to journals, 73.2% (n = 101) were original articles.

Table 1.

Characteristics of the Manuscripts Analyzed (N = 302)

Variable	n (%)
Type of manuscript
Original article	204 (67.5)
Letters to the editor	37 (12.3)
Case report letters	28 (9.3)
Review article	21 (7.0)
Brief communication	8 (2.6)
Commentary	3 (1.0)
Practical psychotherapy	1 (0.3)
Type of rejection
Desk rejection	134 (44.4)
Post peer review rejection	115 (38.1)
Post editorial re-review rejection	53 (17.5)
Reason for rejection
Fatal flaws	230 (76.2)
Nonfatal flaws	72 (23.8)
Type of first author
Psychiatrist	145 (48)
Non-psychiatrist	157 (52)
Origin of papers
India	248 (82.1)
Outside India	54 (17.9)
Prior presentations
Yes	40 (13.2)
No	262 (86.8)
Published elsewhere after rejection at IJPM
Yes	139 (46.0)
No	163 (54.0)

IJPM, Indian Journal of Psychological Medicine.

Figure 1. Flowchart Depicting Results of the Literature Search.

Characteristics of the Journals in Which the Manuscripts Were Published

Of the articles published in a journal (n = 138), 12 (8.7%) were published in predatory journals (Table 2). A considerable proportion was published in journals indexed in MEDLINE (n = 37, 26.8%) or Scopus (n = 53, 38.4%). The quartile distribution of the Scopus-indexed journals in which the articles were published was as follows: 38 (71.7%) in lower two quartiles (Q4: n = 13, 24.5%, Q3: n = 25, 47.2%), and the rest 15 (28.3%) were in upper quartiles (Q2: n = 13, 24.5%, Q1: n = 2, 3.8%).

Table 2.

Characteristics of Journals in Which the Rejected Manuscripts Were Eventually Published (N = 138)

Variable	n (%)
Type of journal
Originating within India	73 (52.9)
Originating outside India	65 (47.1)
On Beall’s list
Yes	12 (8.9)
No	126 (91.1)
On Kscien’s list
Yes	19 (13.8)
No	119 (86.2)
Indexed in MEDLINE
Yes	37 (26.8)
No	101 (73.2)
Indexed in Scopus
Yes	53 (38.4)
No	85 (61.6)
Impact factor available
Yes	26 (18.8)
No	112 (81.2)
CiteScore available
Yes	53 (38.4)
No	85 (61.6)
CiteScore higher than IJPM*
Yes	18 (34.0)
No	35( 66.0)

IJPM, Indian Journal of Psychological Medicine.

Note: *CiteScore was available for only 53 journals and hence the proportions were calculated from these 53.

A total of 18 (13.0%) manuscripts were published in journals with CiteScore and Scopus percentile score higher than IJPM. No significant association was noted between reasons for rejection (fatal vs. nonfatal) and CiteScore of the journal where articles were eventually published (higher vs. lower than IJPM) (χ ² = 0.01, P = 0.95).

Top 10 Journals with Higher Standing than IJPM that Accepted the Papers

The two journals with the highest CiteScores in this group were Q1 journals; seven others were Q2 journals, and one was a Q3 journal. The reasons why IJPM had rejected the 15 articles published in these journals were examined individually (Table 3). Twelve articles were rejected because of fatal flaws such as issues with study design or because they were out of journal scope, lacked sufficient novelty, or deemed a salami/duplicate publication. Only three articles in the list were rejected because of nonfatal flaws. This included poor synthesis of findings for a review article while two original research articles were rejected because of poor presentation and inadequate control of confounders, respectively.

Table 3.

Top 10 Journals of Better Standing (by CiteScore 2019*) that Accepted Manuscripts Rejected by the Indian Journal of Psychological Medicine

Name of Journal (CiteScore 2019)	No. of Manuscripts Accepted	Journal Quartile in Scopus	Type of Submission	Reason for Rejection
European Psychiatry (5.60)	1	Q1	Original article	Issues with study design
International Review of Psychiatry (5.60)	1	Q1	Review article	Poor synthesis of findings
Journal of Neuroimmunology (5.10)	1	Q2	Case report letter	Doubts about the primary diagnosis
Journal of Clinical Psychopharmacology (4.20)	1	Q2	Original article	Issues with study design
Psychiatry Research (3.70)	1	Q2	Original article	Salami/duplicate publication
Asian Journal of Psychiatry (2.70)	6	Q2	2 Case report letters 1 Research letter 1 Brief research communication 2 Original articles	Lack of novelty and Salami/duplicate publication Inadequacies with presentation and conclusion Issues with study design Inadequate control of confounders and Salami/duplicate publication
Asian Pacific Journal of Cancer Prevention (2.50)	1	Q2	Original article	Out of scope
Archives of Psychiatric Nursing (2.40)	1	Q2	Original article	Flaws in development and validation of the instrument
Psychopharmacology Bulletin (2.20)	1	Q2	Non-case-report letter	Out of scope
Indian Journal of Pharmacology (2.10)	1	Q3	Original article	Lack of CTRI registration

CTRI, clinical trial registry of India.

Note: *CiteScore 2019 taken from Scopus.

Factors Related to Publication Status

The origin of the manuscript being from outside of India (OR 1.77, 95% CI 1.06–2.97) and being rejected following peer review and editorial re-review (OR 2.41, 95% CI 1.22–4.74) were significantly correlated with eventual publication; other associations examined were not significant (Table 4). The Hosmer–Lemeshow statistic suggested that the model adequately fit the data (χ ² = 11.35, P = 0.18), while the Nagelkerke R ² was 0.082 indicating that the covariates together explained 8.2% of the variance in the outcome.

Table 4.

Multivariable Analysis to Identify Factors Associated with Eventual Publication

Variable	Univariable OR (95% CI)	Multivariable OR (95% CI)†
Reason for rejection	0.95[0.56–1.62]	1.15[0.65–2.04]
Type of rejection	1.83[1.12–2.91]	1.77[1.06–2.97]*
Type of submission	1.48[0.91–2.42]	1.41[0.83–2.39]
Type of the first author	1.09[0.70–1.72]	1.34[0.81–2.20]
Number of authors	1.1[0.99–1.33]	1.10[0.95–1.28]
Origin of manuscript	2.55[1.34–4.88]	2.41[1.22–4.74]*

OR, odds ratio; CI, confidence intervals.

Note: *significant at p < 0.05; Hosmer-Lemeshow goodness-of-fit statistic χ ² = 11.35, p = 0.18; Nagelkerke R ² of model=0.082.

Discussion

The main study finding was that nearly half of the manuscripts rejected at IJPM were eventually published elsewhere. This finding may have several interpretations; from the journal’s perspective, it may mean that the journal receives a much higher number of submissions than it can accommodate, which in turn means that even good articles, at times, cannot be considered, or that the journal’s editors and peer reviewers are making wrong judgment calls and registering “big misses.” Whereas the first is something that the journal cannot control, the second possibility is concerning and calls for a review of its editorial policies. However, our findings may also mean that other journals make wrong judgment calls by publishing flawed manuscripts.

In this regard, the finding that 13% of the rejected manuscripts were published in higher standing journals than IJPM is both reassuring and concerning. From the journal’s perspective, this low figure appears reassuring as it appears to indicate a low percentage of big misses. However, upon deeper analysis, all such articles were rejected with valid reasons, and many of them with fatal flaws that have no scope for correction. Therefore, our findings are also concerning because they indicate the possibility of wrong judgment calls by other journals regarding the publication of rejected manuscripts.

In many respects, our study results were similar to comparable work from other broad, ⁶ narrow,^{7, 8, 13, 14} and super-specialty journals,^{4, 15} all of which have shown that a considerable proportion of rejected manuscripts were eventually published; the numbers ranged from 41% ⁷ to nearly 76%. ¹³ However, findings from a general non-English language journal showed a lower figure of 10.6% ¹⁶ ; possibly, manuscript translation could be a barrier and limit the choice of journals available for resubmissions.

Original articles constituted the bulk of the rejected submissions analyzed, similar to previous studies.^{4, 7} However, we noted no association between the type of manuscript and eventual publication; this was in contrast to previous studies where original articles were more likely to be eventually published than other types of submissions.^{7, 8} There may be more than one reason for this observation: First, because of the greater number of original article submissions, ³ more of them could be out of scope compared to other categories of submissions; this increases the chances that they will find favor in another better-suited journal. Second, authors are more likely to pursue original manuscripts till full publication for fulfilling promotion criteria.

Articles that cleared the initial triaging but were rejected at the peer review or editorial-re-review stages were more likely to be traced to other journals. Several reasons may be involved here; authors may undertake revisions as suggested by peer reviewers, which may enhance the eventual chances of the manuscript; however, it may also mean that the initial triaging is geared to pick up submissions with fatal flaws, which make them less likely to be published elsewhere.

Our findings have implications for the IJPM as well as other psychiatry journals. The observation that less than half of rejected papers found their way into other journals can be interpreted in both ways; either that the glass is half empty or half full. But the finding that less than a fifth of the published papers were placed in journals with higher metrics than IJPM is a broad vindicator of the journal’s editorial policies. We are unable to comment on whether the reviewer comments included in the rejection decision mail were incorporated by authors and whether this led to a higher likelihood of publication in other journals. Examining this may be an indirect marker of the quality of peer reviews and must be a focus in future similar work. It may also provide a guide to prospective authors who are faced with a choice to either overlook reviewer comments for a rejected manuscript or incorporate them before submitting to another journal. Other journals must also periodically track the fate of rejected manuscripts, particularly those that are rejected at the initial triage, to ascertain if important papers are being missed because of selective editorial policies.

It is important to note the limitations inherent in an analysis of this nature. We have used the CiteScore as a proxy indicator to rank journals as higher or lower than IJPM, although alternate metrics of journal impact such as immediacy index ¹⁷ and PageRank algorithm ¹⁸ exist. The low percentage of big misses registered by the journal could be partly ascribed to the possibility of manuscripts being submitted to IJPM after being rejected by better-ranked journals; this itself reduces the likelihood of big misses. Further, publication in a lower ranked journal following rejection may not necessarily reflect journal editorial policies; a better picture may emerge from comparing publication outcomes of submissions to higher, lower, and comparable journals. We allowed for a time gap of only one to two years between rejection and subsequent publication; it is possible that, given more time, some more manuscripts may be eventually published elsewhere. Finally, if the title and keywords of the manuscript were changed considerably following the rejection, it is possible that the published work would go undetected according to our search criteria.

Nevertheless, the study has certain strengths, such as assessing different submissions, examining several associations of potential interest, and following a systematic search strategy to retrieve publications from the literature. We suggest that more editors carry out similar evaluations of their journals in psychiatry; this will help identify changes needed in journal editorial policies and streamline the publication process in psychiatry. Future work should evaluate the fate of rejected manuscripts over different time periods to analyze how changes in editorial policies have impacted on the selection processes of articles for publication.

In conclusion, approximately half of the manuscripts rejected by the IJPM were published in other journals; of these, nearly 90% are published in journals with a CiteScore lower than IJPM. Manuscripts rejected at the initial triaging stage were less likely to be eventually published. These findings support the journal’s current editorial policies. Additionally, our findings should also provide authors with the hope that redirection of manuscripts rejected by specialty psychiatry journals to alternate journals may meet with success. This is because a manuscript rejection decision may not always imply a lack of quality but it may also reflect a journal’s operating constraints.