Uncited research has received much attention due to its extent and impact on scholarship.1 It has been claimed that articles without a single citation do not have any constructive effect on future research and might be a waste of resources.1 Thus, the percent of uncited articles in any given journal could be considered as a measure of abundance of low-impact articles within that journal, very much like the impact factor (IF) that serves as an index of average impact of all published papers in a given journal.
METHODS
We investigated rate of uncitedness in five high-impact journals: the Annals of Internal Medicine (AIM), the British Medical Journal (BMJ), the Journal of the American Medical Association (JAMA), the Lancet, and the New England Journal of Medicine (NEJM). Using Web of Science (WOS),2 we calculated the rate of uncitedness in the aforementioned journals in 1990, 2000, 2010, and 2015. To compare, we also assessed the uncitedness for all general medical journals.
RESULTS
The mean rate of uncitedness at five years after publication progressively improved from 8.3 for the 1990 cohort to 7.1 and 0.7 for the 2000 and 2010 cohorts, respectively (Fig. 1). The same pattern was observed for the mean rate of uncited researches in the first year of publication (46.0, 38.0, 18.9, and 15.5 for 1990, 2000, 2010, and 2015 cohorts, respectively) and in the second year (16.1, 13.4, 2.7, and 1.7, respectively). There were no associations between the journals’ IF or the number of published articles, and the rate of uncited research (Spearman’s rho IF vs. uncitedness − 0.46, P = 0.18; Spearman’s rho articles vs. uncitedness 0.43, P = 0.21).
Figure 1.
Rate of uncited articles in the five high-impact journals (anonymized as A–E) based on the published articles in 1990, 2000, 2010, and 2015. IF, impact factor; #, number of. Figure 1 and 2 contains poor quality of text inside the artwork. Please do not re-use the file that we have rejected or attempt to increase its resolution and re-save. It is originally poor, therefore, increasing the resolution will not solve the quality problem. We suggest that you provide us the original format. We prefer replacement figures containing vector/editable objects rather than embedded images. Preferred file formats are eps, ai, tiff and pdf. "Figures 2 please confirm if captured correctly.Thank you very much. We provided the highest possible quality of the figures in pdf and tiff formats.
Figure 2 a shows the association between the IF and uncitedness. Overall, there was a negative correlation, in which journals with greater IF had less uncitedness profile. However, this association was nonlinear: the uncitedness dropped sharply in the IF range of 0 to 1, then decreased with less slope for the IF range of 1 to 10, and almost reached a near-zero plateau for the journals with IF > 10. Figure 2 b and c show the longitudinal uncitedness profile for the 2000 and 2010 cohorts, respectively. Top medical journals showed a great leap toward less uncitedness in 2010 cohort compared with 2000, while other journals did not significantly improve
Figure 2.
a The association between the impact factor (IF) and uncitedness. The longitudinal uncitedness profile for the 2000 (b) and 2010 (c) cohorts was divided by the range of IF. #, number of.
DISCUSSION
The overall rate of uncited research was low for all top general medical journals during the above mentioned years, except for one journal in 1990. This could be attributed to the individual policy of the journal or less likely misclassification of uncitable papers into citable articles via the WOS system. Providing unintentional self-citation for original articles by encouraging editorial or correspondence pieces for almost all articles might contribute to the very low rate of uncitedness.
We observed significant reduction in the rate of final uncitedness throughout the decades. This finding might have resulted from having more opportunity to access articles in the age of the Internet or due to fast-growing number of journals in medicine (e.g., 47.6% increase in the number of journals in the category of internal/general medicine), both of which could have enhanced the possibility of citation for an article.
A high IF actually reveals the presence of high-impact articles in a giving journal, rather than the absence or scarcity of low-impact articles. We propose that the uncitedness is more sensitive than IF for measuring the low-impact researches within a journal. Generally, journals with higher IF have lower percentage of uncitedness. However, this association cannot always be reproduced within a narrow category of journals with similar IF. Therefore, uncitedness might find its own way to classify the quality of journals, which could be elucidated later.
As a limitation, our results should be interpreted with caution, since it relied on the WOS. Although indexing databases are accurate on the basis of each source’s criteria,3 there are some discrepancies between them, especially on the citation counts.4 In addition, this study was conducted on general medical journals with special focus on the 5 selected high-impact journals. Therefore, the results of this article might not be generalized to the journals in other disciplines.
Acknowledgments
The authors wish to thank Mr. H. Argasi at the Research Consultation Center (RCC) of Shiraz University of Medical Sciences for his invaluable assistance in editing this manuscript.
Authors’ Contribution
The authors contributed equally according to the ICMJE authorship criteria.
Data Availability
The datasets of the current study are available from the corresponding author on reasonable request.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Ethical Approval
Not applicable
Informed Consent
Not applicable
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Van Noorden R. The science that’s never been cited. Nature. 2017;552(7684):162–4. doi: 10.1038/d41586-017-08404-0. [DOI] [PubMed] [Google Scholar]
- 2.2017 Journal Impact Factor, Journal Citation Reports (Clarivate Analytics, 2018).
- 3.Nowroozzadeh MH. Bibliometrics: Database differences not citation errors. Nature. 2013;503(7476):342. doi: 10.1038/503342d. [DOI] [PubMed] [Google Scholar]
- 4.Plana NM, Massie JP, Bekisz JM, Spore S, Diaz-Siso JR, Flores RL. Variations in databases used to assess academic output and citation impact. N. Engl. J. Med. 2017;376(25):2489–91. doi: 10.1056/NEJMc1616626. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The datasets of the current study are available from the corresponding author on reasonable request.