Abstract
Introduction:
Nowadays, publishing highly-cited papers is important for researchers and editors. In this evidence-based study, the factors influencing the citability of published papers in the field of medicine have been identified.
Material and Methods:
200 papers indexed in Scopus (in two groups: highly-cited and lowly-cited) with 100 papers in each were studied. Needed data were manually collected with a researcher-made checklist. Data analysis was done in SPSS using descriptive and inferential statistics.
Results:
Variables such as journal IF, journal rank, journal subject quartile, the first/corresponding author’s h-index, the number of documents produced by the first/corresponding author, SJR and SNIP had significantly positive correlation with paper citability (p< .05). Other variables, including among others, paper age, paper type, the number of references, the number of authors, indexing institute and journal kind had not any relationship with paper citability (p> .05).
Conclusion:
the factors affecting the citability are among indicators relating to authors, publishing journals and published papers. Determining the extent to which these factors influence the citability of a paper needs further large-scaled research. Authors and editors searching for high-citedness should consider these factors when authoring and publishing papers.
Keywords: Citation, Citability papers, Medicine Evidence-based study
1. INTRODUCTION
Issues on science nature, science classification and science structure have longer been considered and due to the vast development of science and technology in recent years, these issues have been developed and deepened gradually and named under “science studies”. Science studies have two main aspects: qualitative and quantitative aspects. The former considers the philosophy, history and sociology of the science. The latter studies the measurable aspects of science named scientometrics (1).
As one of the methods for evaluating the scientific activities and research management, scientometrics is based on four main variables, including authors, scientific publication, citations and references (2). Most scientometric indicators are based on the citation, as a basic scientific indicator (3). It has been argued in the scientific societies that whether or not the citation can be a good scientific indicator. However, it cannot be ignored in evaluating scientific output as it considers both quality and quantity of scientific output.
Citation connects scientific works and determines their scientific value. Citation is used as an indicator for evaluating the scientific development of scientific associations and fields and the scientific value of published papers (4). By connecting scientific papers, citation makes researchers inform of the previously-published related works (5). Such connection shows the relatedness of the paper at hand with previously published papers in the scientific network (6). Citation is a scientific indicator for measuring the scientific quality of any research paper, too (7).
Many scientometric studies have been conducted in Iran and other countries. Literature review shows that since scientific indicators such as impact factor (IF) and h-index are used for citation analyses by most scientific associations, their completeness and accuracy is challengeable. In other words, suggesting other complementary indicators during recent years clearly shows that main scientific indicators are problematic in depicting the whole picture of the scientific production.
There is no research inclusively conducted on the factors affecting the citability of scientific papers. In few researches, the effect of some variables on papers’ citability has been considered (8). showed that the journal impact factor is not at work in citation rate of papers, especially those with high-prestige authors. Country contribution does increase the citation rates (9, 10). Hopewell and Clarke found that high-reprint articles of the Lancet received more citations than smaller-reprint articles of the journal (11). Yegros-Yegros et al. argued that interdisciplinary articles encounter a challengeable dualism in which multidisciplinary creates a creative work as well as weak work as to its content and scientific value, both affecting the citation rates (12). Multi-authored papers received more citations than single-authored papers (13).
The co-operation among authors and contributing institutions has a significant relationship with citation rate. More the authors interest in collaboration, more their papers receive citations from other papers (14). A study on papers published by FAO revealed that open-accessed journals received more citations than non-open-accessed journals (15). Open-access is one of ways for accessing millions of papers published in many scientific conferences worldwide (16). Davis and Fromerth studied 2,756 papers published in four mathematical journals during 1997- 2005 and found that papers archived in ArXive were cited 25% higher than ones did not accessed in it, especially in case of highly-cited papers (17). For investigating the high-quality of papers published in peer-reviewed journals (18), studied the rates of citations received by accepted vs. rejected manuscripts in these journals, but published by other journals. They found that the accepted papers in pee-reviewed journals received more citations and concluded that these journals could select appropriate papers for publication. In a study on 300 top papers published in economic journals, Moosa found that multi-authored papers have not significantly received more citations than single-authored ones (19).
Each researcher studied one or two variables affecting the citability of scientific papers. No researcher has comprehensively studied the factors affecting the citability. In this study, we aimed to identify the factors affecting the citability of papers in medicine based on the evidence-based approach and answer the following questions:
Does the age of a paper affects its citability?
Does the type of a paper (original research, review, etc.) affect its citability?
Do the scientometric indicators (such as IF, journal rank and SNIP) of a paper affect its citability?
Does the number of contributing countries in authoring a paper affect its citability?
Does the number of references of a paper affect its citability?
Does the h-index of the first / corresponding author of a paper affect its citability?
Dos the accessibility of a paper (open-accessed vs. non-open-accessed) affect its citability?
Does the indexing institute of a paper affect its citability?
2. MATERIAL AND METHODS
In July 2016, a non-restricted search was done in Scopus under the keyword “medicine” and 878,615 papers were retrieved. As in some scientometric indicators such as IF, at least two years of interval is needed for receiving citations, the searched results were refined and the years 2015-2016 were excluded. The retrieved papers were ranked by their citation count. The first 100 highly-cited papers were identified as Group 1. Considering the restriction of the search engine in showing retrieved records (maximum 2000 records), 100 papers at the end of the list were selected as lowly-cited papers and identified as Group 2.
A researcher-made checklist was used for data collection. It included 19 variables in three main groups:
Variables relating to the author(s), including the country of the first/corresponding author, the number of contributing countries, the h index of the first/corresponding author, the number of papers published by the first/corresponding author;
Variables relating to the paper, including the number of citations, paper age, the number of references, the number of authors, and paper type;
Variables relating to publishing journals, including its kind (open-access or non-open-access), IF, IF without self-citation, journal quartile, journal rank, indexing institute, SNIP and SJR.
As the information on journal rank and IF was not accessible via Scopus, all journals publishing the 200 papers of groups 1 and 2 was searched in JCR. Scopus citation report was used for calculating the number of the first/corresponding author’s h index and published papers. When the corresponding author was not mentioned, the first author conceived as the corresponding author. For determining whether or not the studied papers were indexed in other databases including WoS and PubMed, all of them were searched in these two citation databases. If a paper had a PMID in its bibliographic information, it was not searched in PubMed.
All data needed for completing the checklist were collected manually and direct observation. After data collection, the variables were defined in SPSS and needed data were entered. Some approaches of descriptive and inferential statistics were used for data analysis.
3. RESULTS
The mean rate of citations received by the studied papers was 1594.27±1816.61. The mean age of the papers was 14.66±9.49 years. The mean IF of publishing journals was 19.48±17.69. The mean rates of h-indices belonged to the first author and the corresponding author were 47.82±32.88 and 54.65±35.41, respectively. Table 1 shows the descriptive statistics of the studied variables.
Table 1. Descriptive statistics of the studied variables in all studied papers.
| Cahracteristic | N | Minimum | Maximum | Mean±SD |
|---|---|---|---|---|
| Received Citations | 200 | 322 | 12607 | 1594.27±1816.61 |
| Paper Age | 200 | 2 | 51 | 14.66±9.49 |
| References | 177 | 2 | 1478 | 115.20±173.37 |
| Authors | 196 | 1 | 83 | 9.19±12.18 |
| Impact Factor (IF) | 193 | .31 | 137.57 | 19.48±17.69 |
| IF without Self-citation | 193 | .31 | 137.11 | 18.87±17.52 |
| Journal Rank | 193 | 1.12 | 207.25 | 12.03±22.76 |
| First/Corresponding Author’s Country | 195 | 1 | 15 | 1.77±2.12 |
| First Author H-Index | 196 | 1.00 | 185.00 | 47.82±32.88 |
| Corresponding Author H-Index | 196 | 1.00 | 185.00 | 54.65±35.41 |
| First Author’s Docs | 196 | 1 | 2263 | 211.60±268.60 |
| Corresponding Author’s Docs | 196 | 1 | 2263 | 245.81±274.70 |
| SJR | 191 | .10 | 32.24 | 8.22±7.26 |
| SNIP | 190 | .26 | 265.00 | 6.31±19.45 |
As shown in Table 2, 188 (94%) papers were published in non-open-accessed journals. 60%, 24%, and 11% of corresponding authors were from America, Africa and Asia, respectively. Most first authors were from America and Africa. 54% and 43.5% of the papers were original research and review articles, respectively.
Table 2. Frequency (%) of quantitative indicators of the study.
| Cahracteristic | variable | Frequency (%) |
|---|---|---|
| Journal Kind | open access | 12(6) |
| non-open access | 188(94) | |
| Corresponding Author’s Country | America | 120(60) |
| Europe | 49(24.5) | |
| Asia and other | 11(5.5) | |
| Q | Q1 | 173(86.5) |
| other | 20 (10) | |
| First Author’s Country | Usa | 135(67.5) |
| Europe | 48(24) | |
| Asia and others | 13(6.5) | |
| Paper Type | review | 87(43.5) |
| original | 108(54) | |
| other | 5(2.5) | |
| Indexing Institute | PubMed | 8(4) |
| ISI | 4(2) | |
| both | 188(94) |
For determining the possible relationship of citation rate with variables such as journal type and the first/corresponding author’s country, journal subject quartile was used. K2 test was used for determining the possible relationship between citation rate and the corresponding author’s country. Fisher’s exact test was used for determining the possible relationship between the citation rate and paper type. Mann-Whitney test was used for comparing the citation rates of the two paper groups as to variables such as paper age, the number of co-authors, the number of references, journal IF, journal IF without self-citation, journal rank, contributing countries, the h indices of the first and/or the corresponding author, the number of documents produced by the first/corresponding author, SNIP and SJR (Table 3).
Table 3. Comparing characteristics of the two paper groups (highly-cited vs. lowly-cited).
| Cahracteristic | variable | High citation (N=100) | Low citation (N=100) | p-value |
|---|---|---|---|---|
| Journal Type | Open access (%) | 7 (7) | 5(5) | 0.552 |
| Non-open access (%) | 93 (93) | 95 (95) | ||
| Paper Age | Mean±SD (median) | 14.13±18.81(12) | 15.19±10.14(12) | 0.855 |
| References | Mean±SD (median) | 140.77±221.19(53) | 88.75±97.39(62) | 0.548 |
| Authors | Mean±SD(median) | 10.73±13.56(6) | 7.68±10.51(5) | 0.072 |
| Corresponding Author’s Country | Usa(%) | 63(66.3) | 57(67.1) | 0.159 |
| Europe(%) | 29(30.5) | 20(23.5) | ||
| Asia and other(%) | 3(3.2) | 8(9.4) | ||
| IF | Mean±SD(median) | 23.63±15.30(18.39) | 15.28±18.99(8.16) | <0.001 |
| IF without Self- citation | Mean±SD(median) | 22.98±15.23(17.93) | 14.72±18.74(7.74) | <0.001 |
| Journal Rank | Mean±SD(median) | 6.12±9.83(3.15) | 18.00±29.62(5.18) | <0.001 |
| Q | Q1 (%) | 94(96.9) | 79(82.3) | <0.001 |
| other (%) | 3 (3.1) | 17(17.7) | ||
| First Author’s Country | America (%) | 65(67) | 70(70.7) | 0.059 |
| Europe (%) | 29(29.9) | 19(19.2) | ||
| Asia and others (%) | 3(3.1) | 10(10.1) | ||
| Corresponding Author’s Country | Mean±SD (median) | 1.95±2.06(1) | 1.59±2.15(1) | 0.008 |
| Paper Type | review (%) | 45(45) | 42(42) | 0.358 |
| original (%) | 51(51) | 57(57) | ||
| other (%) | 4(4) | 1(1) | ||
| Indexing Institute | PubMed (%) | 3(3) | 5(5) | 0.894 |
| ISI (%) | 2(2) | 2(2) | ||
| both (%) | 95(95) | 93(93) | ||
| First Author’s H-Index | Mean±SD (median) | 57.39±36.80(51) | 38.45±25.41(36) | <0.001 |
| Corresponding Author’s H-Index | Mean±SD (median) | 65.16±36.80(63) | 44.36±31.25(38) | <0.001 |
| First Author’s Docs | Mean±SD (median) | 249.46±270.62(163) | 174.51±262.69(103) | 0.015 |
| Corresponding Author’s Docs | Mean±SD (median) | 292.25±272.35(212) | 200.31±270.68(136) | <0.001 |
| SJR | Mean±SD (median) | 10.05±7.51(6.99) | 6.34±6.52(3.69) | <0.001 |
| SNIP | Mean±SD (median) | 8.4±26.68(4.91) | 4.18±5.73(2.5) | <0.001 |
As Table 3 shows, journal type, paper age, number of references, number of authors, paper type, and indexing institute had not any significant relationship with citation rates of the papers (p>.05). The other variables, including among others, journal IF, journal rank, journal quartile, h index of the first/corresponding authors, the number of documents produced by the first/corresponding author, SJR and SNIP had significantly positive relationship with the citation rates of the studied papers (P<0.05).
4. DISCUSSION
Many factors are at work in the citability of scientific papers that studying them are important as to their influence on the citation rate. These factors may be different in scientific disciplines. In the field of medicine, it appears that the kind of journal access (open access vs. non-open access) does not affect the citability. The possible reasons of the finding needs further research as previous research showed a positive relationship between open access papers and their citability in some non-medical fields (15, 20, 21).
There was no relationship between paper age and citation rate. This depends on the nature of scientific disciplines and fields. The reason of the finding may be that because of ever-changing nature of information in medicine, there is low tendency to cite old papers and authors tend to cite newly-published works in the field (7).
In this study, there was no significant relationship between the number of references and that of citation received. The possible relationship is challengeable and some researchers argue that more references in a paper do not mean that it will receive high citation (7). Studying co-authorship and co-citation relations can be beneficial to this aspect.
We found that the number of authors had not any effect on paper citability. It is expected that the quality of multi-authored papers is high and they receive more citations. As there is contradictory findings in this aspect (22, 23), achieving appropriate results needs further research.
There was not any citation bias in citing papers published in different geographical areas in this study. Factors other than geographical one can be effective in citing medical papers. This finding is not in line with the finding that more productive countries receive more citations (24).
There was a significant relationship between paper citability and publishing journal IF-with or without taking self-citations into account. This finding is justifiable considering the definition and functionality of IF (25). Considering a significantly positive relationship between paper citability and journal rank, it can be said that this relationship reflects the function of IF as high IF means high journal rank (26). Similar to our study, García et al. found that papers published in the journals included in the first subject quartile had more citation rate (27).
Since we did not find any significant relationship between paper citability and paper type in medical field, other researcher found that review articles receive more citations due to their nature (28).
As most studied papers were indexed in both ISI and PubMed, there was no relationship between paper citability and their being indexed in each of these citation databases. Another study can be done for comparing papers indexed in ISI vs. PubMed.
We found a significant relationship between paper citability and the first/corresponding author h-index. As h-index manifests authors’ ability to being cited, the co-authorship of the first/corresponding author with authors with high h-indices can increase the citability of documents co-authored (29).
The positive relationship was seen between the number of documents produced by the first/corresponding author and the number of citations received by his/her paper at hand. However, the number of citations received by previously published papers of the first/corresponding author did not studied here. It is notable that in different scientific fields, multi-authored papers receive more citations than single-authored papers (30).
5. CONCLUSION
Our study confirmed the relationship between paper citability and SJR on one hand and SNIP on the other hand. It can be said that both indicators are disciplinary-based and have high visibility. Subject connection and visibility are of factors affecting the citability in this regard.
Determining and explaining the factors affecting citability of papers published in the field of medicine are important for determining highly-cited authors, works and journals in the field. Ones administrating medical research can use these criteria in macro and micro levels. It is appeared that citability factors at work in author, paper and journal levels and clearly defining them and accurately measuring their influence need further large-scaled researches in the field.
Authors contributions:
Mousa Yaminfirooz were the responsible for the study design, data analysis, interpretation of results, and drafting the manuscript. Farzaneh Raeesi Ardali contributed to the data analysis and revision of the manuscript. All authors read and approved the final manuscript.
Conflict of interest:
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
REFERENCES
- 1.Hood W, Wilson C. The literature of bibliometrics, scientometrics, and informetrics. Scientometrics. 2001;52(2):291–314. [Google Scholar]
- 2.Yaminfirooz M, Gholinia H. Multiple h-index: a new scientometric indicator. The Electronic Library. 2015;33(3):547–56. [Google Scholar]
- 3.Ahangar HG, Siamian H, Yaminfirooz M. Evaluation of the scientific outputs of researchers with similar H index: A critical approach. Acta Inform Med. 2014;22(4):255–8. doi: 10.5455/aim.2014.22.255-258. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Tirgar A, Yaminfirooz M, Ahangar HG. Subject Sameness Index: a new scientometric indicator. European Science Editing. 2013;39(1):3–4. [Google Scholar]
- 5.Yaminfirooz M, Motallebnejad M, Gholinia H, Esbakian S. Quantitative and qualitative evaluation of Iranian researchers’ scientific production in dentistry subfields. Acta Inform Med. 2015;23(5):301–5. doi: 10.5455/aim.2015.23.301-305. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Moed HF. Springer Science & Business Media; 2006. Citation analysis in research evaluation. [Google Scholar]
- 7.Bornmann L, Daniel H-D. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. 2008;64(1):45–80. [Google Scholar]
- 8.Prathap G, Mini S, Nishy P. Does high impact factor successfully predict future citations? An analysis using Peirce’s measure. Scientometrics. 2016;108(3):1043–7. [Google Scholar]
- 9.Rousseau R, Ding J. Does international collaboration yield a higher citation potential for US scientists publishing in highly visible interdisciplinary Journals? Journal of the Association for Information Science and Technology. 2015.
- 10.Smith MJ, Weinberger C, Bruna EM, Allesina S. The scientific impact of nations: journal placement and citation performance. PloS one. 2014;9(10):e109195. doi: 10.1371/journal.pone.0109195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hopewell S, Clarke M. How important is the size of a reprint order? International journal of technology assessment in health care. 2003;19(04):711–4. doi: 10.1017/s0266462303000680. [DOI] [PubMed] [Google Scholar]
- 12.Yegros-Yegros A, Rafols I, D’Este P. Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PloS One. 2015;10(8): e0135095. doi: 10.1371/journal.pone.0135095. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Persson O, Glänzel W, Danell R. Inflationary bibliometric values: The role of scientific collaboration and the need for relative indicators in evaluative studies. Scientometrics. 2004;60(3):421–32. [Google Scholar]
- 14.Figg WD, Dunn L, Liewehr DJ, Steinberg SM, Thurman PW, Barrett JC, et al. Scientific collaboration results in higher citation rates of published articles. Pharmacotherapy: The Journal of Human Pharmacology and Drug Therapy. 2006;26(6):759–67. doi: 10.1592/phco.26.6.759. [DOI] [PubMed] [Google Scholar]
- 15.Kousha K, Abdoli M. The citation impact of Open Access Agricultural Research: a comparison between OA and Non-OA publications. Online Information Review. 2010;34(5):772–85. [Google Scholar]
- 16.Gargouri Y, Hajjem C, Larivière V, Gingras Y, Carr L, Brody T, et al. Self-selected or mandated, open access increases citation impact for higher quality research. PloS One. 2010;5(10):e13636. doi: 10.1371/journal.pone.0013636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Davis P, Fromerth M. Does the arXiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics. 2007;71(2):203–15. [Google Scholar]
- 18.Bornmann L, Daniel HD. Selecting manuscripts for a high-impact journal through peer review: A citation analysis of communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology. 2008;59(11):1841–52. [Google Scholar]
- 19.Moosa IA. Citations, journal ranking and multiple authorships: evidence based on the top 300 papers in Economics. Applied Economics Letters. 2017;24(3):175–81. [Google Scholar]
- 20.Craig ID, Plume AM, McVeigh ME, Pringle J, Amin M. Do open access articles have greater citation impact?: a critical review of the literature. Journal of Informetrics. 2007;1(3):239–48. [Google Scholar]
- 21.Kurtz MJ, Eichhorn G, Accomazzi A, Grant C, Demleitner M, Henneken E, et al. The effect of use and access on citations. Information Processing & Management. 2005;41(6):1395–402. [Google Scholar]
- 22.Haslam N, Ban L, Kaufmann L, Loughnan S, Peters K, Whelan J, et al. What makes an article influential? Predicting impact in social and personality psychology. Scientometrics. 2008;76(1):169–85. [Google Scholar]
- 23.Leimu R, Koricheva J. Does scientific collaboration increase the impact of ecological articles? BioScience. 2005;55(5):438–43. [Google Scholar]
- 24.Pasterkamp G, Rotmans J, de Kleijn D, Borst C. Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles. Scientometrics. 2007;70(1):153–65. [Google Scholar]
- 25.McVeigh ME, Mann SJ. The journal impact factor denominator: defining citable (counted) items. Jama. 2009;302(10):1107–9. doi: 10.1001/jama.2009.1301. [DOI] [PubMed] [Google Scholar]
- 26.Falagas ME, Alexiou VG. The top-ten in journal impact factor manipulation. Archivum immunologiae et therapiae experimentalis. 2008;56(4):223–6. doi: 10.1007/s00005-008-0024-5. [DOI] [PubMed] [Google Scholar]
- 27.García JA, Rodriguez-Sánchez R, Fdez-Valdivia J, Martinez-Baena J. On first quartile journals which are not of highest impact. Scientometrics. 2011;90(3):925–43. [Google Scholar]
- 28.Garfield E, Welljams-Dorof A. Citation data: their use as quantitative indicators for science and technology evaluation and policy-making. Science and Public Policy. 1992;19:321. [Google Scholar]
- 29.Costas R, Bordons M. The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. Journal of informetrics. 2007;1(3):193–203. [Google Scholar]
- 30.Wuchty S, Jones BF, Uzzi B. The increasing dominance of teams in production of knowledge. Science. 2007;316(5827):1036–9. doi: 10.1126/science.1136099. [DOI] [PubMed] [Google Scholar]