Abstract
Purpose
We classified the articles published in the journal Hip & Pelvis and analyzed the relationship between study characteristics and citation rates.
Materials and Methods
All articles published in Hip & Pelvis from 2009 to 2019 were included. We classified the articles according to the type, language, listing in PubMed Central (PMC), treatment modality, material, design, anatomical focus, number of authors, and number of cases. We analyzed the citation rate according to this classification, with yearly citation rate reflecting the exposure period until March 2020.
Results
The yearly citation rate increased significantly after the language of the journal was changed from Korean to English in June 2014 (mean=0.96 vs. 1.63, P<0.05), and again after the journal was listed in PMC in March 2016 (mean=1.05 vs. 1.92, P<0.05). The yearly citation rates of review articles was highest, followed by those of editorials, original articles, and case reports (in this order). Among original articles, trauma-related articles had higher yearly citation rates than non-trauma-related articles (mean=1.00 vs. 0.68, P=0.034). Among clinical articles, studies focusing on the pelvis had higher yearly citation rates than studies on the hip or femur (mean=1.85 vs. 0.71 vs. 0.91, P=0.003).
Conclusion
The yearly citation rate of articles increased significantly after the language of Hip & Pelvis was changed to English and after the journal was listed in PMC. The mean yearly citation rate of articles focusing on the pelvis was significantly higher than that of articles focusing on the hip or femur.
Keywords: Hip & Pelvis, Citation rate, Type, Language, PubMed
INTRODUCTION
Most scholars aim to contribute to the improvement of healthcare by reporting studies that are read by experts worldwide. Therefore, publishing studies in popular journals is desirable. One indicator of how many readers an article has reached is the citation rate1,2). While the citation rate does not indicate the quality of a published article, a high citation rate of a journal provides indirect evidence of the number of readers1,3,4). Several indicators have been developed to reflect the number of citations of international journals, such as the h-index, Source Normalized Impact per Paper (SNIP), and SCImago Journal Rank (SJR). These indicators are typically calculated annually along with a journal's impact factor and are regularly published by the journal5,6,7,8).
The Korean Medical Citation Index (KoMCI) project was started in 2000 and provides the number of citations of Korean journals in domestic or international journals. The data can be found on the KoMCI website (https://komci.org) and the National Research Foundation website (https://www.kci.go.kr). The number of citations is used to select journals or candidates that can be registered with the National Research Foundation. Recently, some Korean journals have changed their publication language to English in order to increase the number of readers and citations in international journals. The Korean orthopedic journals that have recently made this transition include the Journal of the Korean Hip Society, Journal of Korean Knee Society, Journal of Bone Metabolism, and Journal of the Korean Shoulder and Elbow Society. The Journal of the Korean Hip Society was converted into an English-language journal and renamed as Hip & Pelvis) in June 2014 and was listed in PubMed Central (PMC), signifying recognition as an international journal, in March 2016. These changes allowed for articles that were previously cited only by domestic researchers to also be cited by international researchers.
The citation characteristics of articles published in Hip & Pelvis have not been reported. In this study, which was based on the data provided by KoMCI, the citation rate of Hip & Pelvis in both domestic and international journals and the characteristics of the cited articles were reviewed to determine whether authors can expect a high number of citations when publishing in Hip & Pelvis.
MATERIALS AND METHODS
All articles published in Hip & Pelvis from 2009 to 2019 were included in this study. Using the abstracts and full text, two independent reviewers classified the articles according to article type, language, listing in PMC, treatment modality, material, time of study, study design, anatomical focus, number of authors, and number of cases (Fig. 1). The article types included original article, case report, review article, editorial, and letter to the editor. The articles were also categorized according to material as clinical research, experimental studies, or general orthopedics. Clinical research articles were further classified according to anatomical focus (hip, pelvis, or femur) The articles were also classified according to time of study (retrospective or prospective), study design (case series, case control study, randomized controlled trial [RCT], or study of a measurement tool), and treatment modality (hip joint replacement, arthroscopy, internal fixation after trauma, or unclassified). We analyzed the citation rates depending on the classification, with yearly citation rate reflecting the exposure period until March 2020. The number of citations was determined using Synapse data provided by KoMCI, Synapse, Crossref, and Google Scholar duplicate citations were excluded9).
Fig. 1. Classification according to article type, material, anatomical focus, and study design.
1. Statistical Analysis
All statistical analyses were performed using IBM SPSS software version 19.0 (IBM Corp., Armonk, NY, USA). For nominal categorical variables (article type, language, PMC listing, treatment modality, material, design, anatomical focus), the number of citations was analyzed using the Kruskal-Wallis test and independent-sample t-test. For continuous variables (number of authors, number of cases), the number of citations was analyzed using linear regression analysis. Each variable was then analyzed for correlation with the number of citations using multiple regression analysis6,8). Statistical significance was set at P<0.05, and when a significant probability was found in the Kruskal-Wallis test, a post hoc analysis was performed with the significance level corrected by the Bonferroni method.
RESULTS
From 2009 to 2019, Hip & Pelvis published 492 articles, including 277 original articles (56.3%), 117 case reports (23.8%), and 78 reviews (15.9%). The average yearly citation rate was 0.83 (standard deviation=2.13). A total of 295 articles (60.0%) were cited more than once, and the most cited article was “Management of Periprosthetic Joint Infection”10) published in 2018 (38 citations per year). After the change of the publication language to English in June 2014, the yearly citation rate increased significantly (mean=0.96 vs. 1.63, P<0.05) (Table 1). After the journal was listed in PMC in March 2016, the yearly citation rate further significantly increased (mean=1.05 vs. 1.92, P<0.05) (Table 2). The highest average yearly citation rate was 2.23 in 2018, at (Fig. 2).
Table 1. Yearly Citation Rates According to Article Language.
| Variable | Number of articles | Mean number of yearly citations | P-value |
|---|---|---|---|
| Korean | 257 | 0.96±0.16 | <0.05 |
| English | 235 | 1.63±2.86 | |
| Total | 492 |
Values are presented as number only or mean±standard deviation.
Table 2. Yearly Citation Rates Before and After PubMed Central (PMC) Listing.
| Variable | Number of articles | Mean number of yearly citations | P-value |
|---|---|---|---|
| Before PMC listing | 78 | 1.05±0.96 | <0.05 |
| After PMC listing | 157 | 1.92±3.40 | |
| Total | 235 |
Values are presented as number only or mean±standard deviation.
Fig. 2. Average yearly citation rate.
The most cited article type was review article, followed by editorial, original article, and case report (in this order). The yearly citation rate of case reports was significantly lower than that of review articles (P=0.024) or original articles (P<0.001); however, it was not significantly lower than that of editorials (Table 3). Among original articles, clinical research articles were the most cited, followed by general orthopedics articles and experimental studies; however, there was no significant difference in the citation rate between the materials of original articles (Table 4). Only retrospective clinical studies were published in the case report category, and there were no statistically significant differences in the yearly citation rate of case reports based on the number of authors, the number of cases, anatomical focus, or whether the articles reported traumatic or non-traumatic cases (Table 5). The most published treatment modality was arthroplasty, followed by internal fixation, unclassified modalities, and arthroscopy (in this order); however, there were no significant differences in the yearly citation rate among treatment modalities (Table 6).
Table 3. Yearly Citation Rates by Article Type.
| Variable | Number of articles | Mean number of yearly citations | P-value |
|---|---|---|---|
| Original article | 277 | 0.82±1.25 | <0.001 |
| Case report | 117 | 0.34±0.57 | Reference |
| Review article | 78 | 1.54±4.59 | 0.024 |
| Editorial/letter to editor | 20 | 1.08±1.75 | 0.797 |
| Total | 492 |
Values are presented as number only or mean±standard deviation.
Table 4. Study Characteristics Associated with the Yearly Citation Rates of Original Articles.
| Variable | Number | Mean number of yearly citations | Univariate | Multivariate (95% CIs) |
|---|---|---|---|---|
| Trauma related | P=0.034* | |||
| Non-traumatic | 155 | 0.68±1.21 | ||
| Traumatic | 122 | 1.00±1.28 | ||
| Time of study | ||||
| Retrospective | 274 | 0.83±1.25 | ||
| Prospective | 3 | 0 | ||
| Number of authors | R2=0.003 | |||
| B=-0.050 | ||||
| P=0.331† | ||||
| Number of cases | R2=0.006 | |||
| B=0 | ||||
| P=0.183† | ||||
| Number of references | R2=0.008 | |||
| B=0.020 | ||||
| P=0.150† | ||||
| Clinical location | P=0.003‡ | |||
| Hip | 182 | 0.71±1.19 | P=0.002 (0.34-1.93) | |
| Pelvis | 15 | 1.85±1.82 | Reference | |
| Femur | 74 | 0.91±1.18 | P=0.023 (0.10-1.78) | |
| Unclassified | 6 | 0.58±0.91 | ||
| Design of study | P=0.198‡ | |||
| Case series | 205 | 0.86±1.25 | Reference | |
| Case control | 54 | 0.82±1.31 | P=0.656 (0.21-1.65) | |
| RCT | 4 | 0 | P=0.430 (0.88-1.71) | |
| Measurement tool | 12 | 0.41±1.05 | P=0.074 (0.10-1.83) |
Values are presented as number only or mean±standard deviation.
CIs: confidence intervals, B: regression coefficient, RCT: randomized controlled trial.
*Student t-test, †Univariate linear regression test, ‡Kruskal-Wallis test.
Table 5. Study Characteristics Associated with the Yearly Citation Rates of Case Reports.
| Variable | Number | Mean citations | Univariate | Multivariate (95% CIs) |
|---|---|---|---|---|
| Trauma related | P=0.240* | |||
| Non-traumatic | 73 | 0.42±0.44 | ||
| Traumatic | 44 | 0.29±0.66 | ||
| Number of authors | R2=0.040 | |||
| B=-0.074 | ||||
| P=0.031† | ||||
| Number of cases | R2=0.000 | |||
| B=-0.022 | ||||
| P=0.855† | ||||
| Clinical location | P=0.442‡ | |||
| Hip | 68 | 0.26±0.48 | Reference | |
| Pelvis | 24 | 0.46±0.66 | P=0.455 (-0.57-0.17) | |
| Femur | 23 | 0.33±0.57 | P=0.939 (-0.40-0.27) | |
| Unclassified | 2 | 1.28±1.57 | P=0.148 (0.87-2.08) |
Values are presented as number only or mean±standard deviation.
CIs: confidence intervals, B: regression coefficient.
*Student t-test, †Univariate linear regression test, ‡Kruskal-Wallis test.
Table 6. Yearly Citation Rates According to Treatment Modality.
| Variable | Number of articles | Mean number of yearly citations | P-value |
|---|---|---|---|
| Arthroplasty | 220 | 0.79±1.46 | 0.070 |
| Arthroscopy | 21 | 0.72±1.33 | |
| Internal fixation | 138 | 0.79±1.15 | |
| Unclassified others | 113 | 0.97±3.70 | |
| Total | 492 |
Values are presented as number only or mean±standard deviation.
Among the original clinical research articles, those focusing on trauma were cited more often than those reporting non-traumatic cases (mean=1.00 vs. 0.68, P=0.034). In addition, original clinical research articles focusing on the pelvis were cited significantly more often (mean=1.85) than those focusing on the hip (mean=0.71) or femur (mean=0.91) (P=0.003). There was no statistically significant relationship between the yearly citation rate and the number of authors, the number of cases, or study design (Table 7).
Table 7. Yearly Citation Rates According to Material of Original Article.
| Variable | Number of articles | Mean number of yearly citations | P-value |
|---|---|---|---|
| Clinical research | 272 | 0.83±1.26 | 0.911 |
| Experimental study | 3 | 0.26±0.23 | |
| General orthopedics | 2 | 0.32±0.06 | |
| Total | 277 |
Values are presented as number only or mean±standard deviation.
DISCUSSION
The conversion of Hip & Pelvis to an English-language journal in June 2014 and its listing in PMC in March 2016 significantly increased its yearly citation rate, likely owing to the inclusion of international authors publishing in Hip & Pelvis and citing the journal's articles. Therefore, these changes have increased international interest in the journal.
Between 2009 and 2019, of the 492 articles published in Hip & Pelvis, the majority were original articles, followed by case reports and reviews. Original articles that focused on trauma or the pelvis were cited significantly more frequently than articles reporting non-traumatic cases or those focusing on the hip or femur. However, the number of articles focusing on the pelvis was low (15/277). These results suggest that original or review articles focusing on the pelvis are most likely to be read and cited among all of the articles in Hip & Pelvis.
The yearly citation rate of case reports was significantly lower than that of original articles or reviews. This may be due to the fact that the average number of references cited in case reports was 10.16, considerably lower than that in original articles (24.83) and in review articles (33.29). However, case reports are important to the improvement of healthcare, as they report the discovery of new diseases, rare forms of common diseases, and rare diseases. Therefore, the submission of case reports should be encouraged even if the number of citations is not as high as that of original or review articles.
Original clinical research articles that focused on trauma had a higher yearly citation rate than those reporting non-traumatic cases, and articles that focused on the pelvis, though fewer in number, had a higher yearly citation rate than those focusing on the hip or femur. We considered articles focusing on peri-acetabular osteotomy for deformity correction or the treatment of acetabular or pelvic fractures as belonging to the category of pelvis articles. Therefore, original, trauma-related articles or studies that focus on the pelvis are most likely to be read and cited among articles published in Hip & Pelvis. Interestingly, only one RCT article was published in Hip & Pelvis between 2009 and 2019 and its yearly citation rate was not significantly different from those of other types of original articles. In recent years, large-scale, well-organized RCT studies were more likely to be submitted to SCI(E) (Science Citation Index Expanded)-level international academic journals. Nevertheless, the submission of RCT articles should be encouraged, as these are difficult to perform and not as common as other types of studies.
This study has several limitations. First, the variation in the number of citations due to the difference in exposure periods after publication was corrected by simply dividing the number of citations by the number of years since publication. However, each journal has different citation patterns by year, which are evaluated using an immediacy index indicating how quickly a published article is cited, or a cited half-life indicating for how long it is cited. Therefore, our method of correcting for the exposure period results in more recently published papers with a short citation half-life having a higher citation index. To minimize this limitation, most research on citation characteristics in international journals includes articles published in a period of less than 18 months4,5,6,7). However, the number of articles published in Hip & Pelvis is relatively small and the average number of citations per year is low, making it difficult to include only data for one publication year. Therefore, the results were analyzed without classification according to the year in this study. Second, the classification of articles by anatomical focus was difficult when articles focused on more than one location. For example, the article “Fracture of the Greater Trochanter during Closed Reduction of Obturator Type Hip Dislocation” can be classified as either a hip or a femur article; in this case we classified it as a hip article. If we classified it as a femur article, this may have affected the analysis of the number of citations. Finally, the number of citations includes those in domestic and international journals but not in published books or on websites. If the number of citations in these media is added, the results of the study may be different. Future studies should apply differentiated weights by year of publication based on characteristics such as the citation half-life in order to reduce the limitation of the exposure period. Future studies should also analyze articles focusing on more than one anatomical location separately and include citations in books and on websites.
CONCLUSION
The yearly citation rate of articles published in Hip & Pelvis increased significantly after the journal's language was changed to English and the journal was listed in PMC. The number of citations for trauma-related articles and articles focusing on the pelvis was significantly higher than that of other articles. These types of articles are most likely to be read and cited in the future, and therefore, may be more likely to be accepted by the journal. The results of this study can be used as a reference to determine which manuscripts should be submitted to Hip & Pelvis and which published articles will be cited the most.
ACKNOWLEDGEMENTS
This study was supported by a 2020 research grant from Pusan National University Yangsan Hospital.
Footnotes
CONFLICT OF INTEREST: The authors declare that there is no potential conflict of interest relevant to this article.
References
- 1.Garfield E, Welljams-Dorof A. Of Nobel class: a citation perspective on high impact research authors. Theor Med. 1992;13:117–135. doi: 10.1007/BF02163625. [DOI] [PubMed] [Google Scholar]
- 2.Garfield E. How can impact factors be improved? BMJ. 1996;313:411–413. doi: 10.1136/bmj.313.7054.411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Eberlin KR, Labow BI, Upton J, 3rd, Taghinia AH. High-impact articles in hand surgery. Hand (N Y) 2012;7:157–162. doi: 10.1007/s11552-011-9388-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Okike K, Kocher MS, Torpey JL, Nwachukwu BU, Mehlman CT, Bhandari M. Level of evidence and conflict of interest disclosure associated with higher citation rates in orthopedics. J Clin Epidemiol. 2011;64:331–338. doi: 10.1016/j.jclinepi.2010.03.019. [DOI] [PubMed] [Google Scholar]
- 5.Miguel-Dasit A, Aleixandre R, Valderrama JC, Martí-Bonmatí L, Sanfeliu P. Hypothetical influence of non-indexed Spanish journals on the impact factor of radiological journals. Eur J Radiol. 2005;54:321–326. doi: 10.1016/j.ejrad.2004.07.017. [DOI] [PubMed] [Google Scholar]
- 6.Kulkarni AV, Busse JW, Shams I. Characteristics associated with citation rate of the medical literature. PLoS One. 2007;2:e403. doi: 10.1371/journal.pone.0000403. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lopez J, Calotta N, Doshi A, et al. Citation rate predictors in the plastic surgery literature. J Surg Educ. 2017;74:191–198. doi: 10.1016/j.jsurg.2016.08.005. [DOI] [PubMed] [Google Scholar]
- 8.Fujihara Y, Fujihara N, Yamamoto M, Hirata H. Citation analysis of articles about hand surgery published in orthopaedic and hand surgery journals. J Hand Surg Asian Pac Vol. 2019;24:36–44. doi: 10.1142/S2424835519500073. [DOI] [PubMed] [Google Scholar]
- 9.Korean Medical Citation Index (KoMCI) Web [Internet] Seoul: KoMCI; 2018. [cited 2020 Jan]. Available from: https://komci.org/KoMCIWeb_gs.php. [Google Scholar]
- 10.Li C, Renz N, Trampuz A. Management of periprosthetic joint infection. Hip Pelvis. 2018;30:138–146. doi: 10.5371/hp.2018.30.3.138. [DOI] [PMC free article] [PubMed] [Google Scholar]