Abstract
Purpose
Determine the impact of self-citation on external funding for orthopedic fellowship directors.
Methods
The San Francisco Match's website identified directors encompassing 8 subspecialties. The Scopus database identified the number of publications, citations, and h-index for each director. H-index was assessed with/without self-citation.
Results
Mean publications, citations, self-citation rate, and h-index for the 446 directors were 71.2, 1816, 3.86%, and 18.3, respectively. Excluding self-citations reduces mean h-index to 18.0; and h-index changed by ≤ 1 integer in 95% of directors.
Conclusions
Self-citation has minimal impact among fellowship directors and should not be adjusted for when considering external funding.
Keywords: H-index, Self-citation, Self-citation, Orthopaedic surgery, Orthopedic surgery, Fellowship, External funding
1. Introduction
The Hirsch Index (h-index) is a simple, and frequently used academic metric first introduced by Jorge Hirsch in 2005.1 The h-index attempts to measure both the quantity and quality of scholarly activity defined as “the number of papers with citation number > or = h.”1 The research productivity of trainees, teaching staff, clinician-researchers, program directors and even countries can be gauged using the h-index.2, 3, 4, 5, 6, 7, 8, 9, 10 Additionally, the h-index has been shown to correlate with differences in industry payment size and distribution among academic surgeons receiving industry research support or NIH funding.11,12 Previous studies have investigated the impact of self-citation among other medical and surgical specialties.13, 14, 15, 16, 17, 18 With regards to orthopedic surgery, the role of self-citation has been examined in orthopaedic surgical residencies and hand surgery fellowship programs and was found to have a minimal impact for most researchers.15,17 To date, however, there has been no examination of the role of self-citation among the eight other orthopaedic subspecialties that offer fellowship training: arthroplasty, foot and ankle, oncology, pediatrics, shoulder and elbow, spine, sports medicine, and trauma. The purpose of this study is to describe the impact and role of self-citation among the fellowship directors of different orthopaedic surgery subspecialties. This information would be beneficial in determining if benchmarks would be useful for both academics and distribution of external funding as it pertains to orthopedic subspecialties.
2. Method and materials
The SF Match website (https://sfmatch.org) lists fellowship positions across eight orthopaedic subspecialties. Each program's subspecialty, number of fellows, and program director were identified. Fellowship programs were categorized as arthroplasty, foot and ankle, oncology, pediatrics, shoulder and elbow, spine, sports medicine, and trauma. The SF match website was accessed on October 16th, 2018.
The Scopus database (https://www.scopus.com) was used to extract the number of publications, total number of citations, and h-index for each respective program director. This database was accessed between the dates of October 16 – November 30, 2018. Author name and academic affiliation were entered into the ‘‘author search’’ page and metrics were available within the “View citation overview” page as previously described by Rad et al.13 The total number of citations and h-index were recorded before and after checking the box labeled “Exclude self-citations of selected author” and any changes were noted.
2.1. Statistical analysis
Program directors were first analyzed in two groups, those that the h-index remained unchanged, and those that changed by one or more integer. Subgroup analysis of all academic metrics was performed with regard to subspecialty type and number of fellows at each program. Qualitative variables were analyzed with a Chi-Square test and continuous variables were analyzed with a Student t-test. Self-citation rate for the cohort and subgroups was calculated as total number of self-citations divided by total number of citations. Comparison of self-citation rates between the subgroups was performed as an analysis of the means with a Student t-test. Data were extracted and analyzed using a Microsoft Excel spreadsheet (Microsoft Corporation, Redmond, Washington).
3. Results
A total of 446 fellowship programs offering 846 fellowship positions were identified. The mean number ± standard deviation (SD) of publications, citations, and h-index for the cohort were 71.2 ± 74.9, 1816 ± 2608, and 18.3 ± 13.7, respectively. The self-citation rate for the entire cohort was 3.86%. Excluding self-citations reduces the mean number of citations to 1746 ± 2463 and h-index to 18.0 ± 14.3. The h-index changed by one integer or less for 95% of program directors (425/446); while it remained completed unchanged in 78% of program directors (349/446). Only 5% (21/446) of program directors had a change in h-index of greater than one integer and only 1% (5/446) of program directors had a change of greater than two integers.
Program directors were first stratified into two groups; those who had a change in h-index and those who did not (Table 1). Program directors were more likely to have a change in h-index if the fellowship program offered more than one position (17% vs. 27%, p = 0.009). Program directors who had a change in h-index had a significantly higher number of overall publications (52 vs. 142, p < 0.001), overall citations (1201 vs. 4029, p < 0.0001), h-index (15 vs. 31, p < 0.0001), and self-citation rate (2.15% vs. 5.70%, p < 0.0001).
Table 1.
Program and author characteristics.
| h-index Unchanged | h-index Changed | p value | |
|---|---|---|---|
| Total (#) | 349 (78%) | 97 (22%) | |
| Specialty Type (#) | |||
| Arthroplasty | 72 (80%) | 18 (20%) | >0.6 |
| Foot and Ankle | 39 (78%) | 11 (22%) | >0.9 |
| Oncology | 15 (79%) | 4 (21%) | >0.9 |
| Trauma | 46 (78%) | 13 (22%) | >0.9 |
| Pediatrics | 35 (83%) | 7 (17%) | >0.4 |
| Shoulder and Elbow | 25 (89%) | 3 (11%) | >0.1 |
| Spine | 50 (72%) | 19 (28%) | >0.2 |
| Sports | 67 (75%) | 22 (25%) | >0.5 |
| Fellows (#) | |||
| 1 Fellow | 189 (83%) | 38 (17%) | <0.009 |
| >1 Fellow | 160 (73%) | 59 (27%) | <0.009 |
| >2 Fellows | 65 (64%) | 36 (36%) | <0.0001 |
| >3 Fellows | 30 (60%) | 20 (40%) | <0.0009 |
| Academic Metrics (mean) | |||
| Number of Publications | 52 | 142 | <0.0001 |
| Number of Citations | 1201 | 4029 | <0.0001 |
| h-index | 15 | 31 | <0.0001 |
| Self-Citation Rate | 2.15% | 5.70% | <0.0001 |
Surgeons were also stratified into different groups based on the magnitude of change in h-index (Table 2). As the absolute change in h-index increased, so did the number of publications, h-index, and individual citations. For the 5% of surgeons with a change of h-index of two integers or more, the mean number of publications, h-index, and citations were 207, 42, and 6798, respectively.
Table 2.
Stratification of Different Groups of Authors Based on h-index Increase after Self-citation.
| h-index did not change | h-index increased 1 integer | h-index increased 2 or more integers | |
|---|---|---|---|
| Number (% of all) | 349 (78%) | 76 (17%) | 21 (5%) |
| Mean Number of Publications (Median; Range) | 52 (35; 0–344) | 124 (104; 12–426) | 207 (180; 99–514) |
| Mean h-Index including Self-citation (Median; Range) | 15 (13; 0–59) | 28 (28; 5–65) | 42 (41; 18–78) |
| Mean Number of Individual Citations (SD) | 1201 (1665) | 3264 (2850) | 6798 (5390) |
| Mean Proportional Increase in H-index from Self-citation by % (Median; Range) | 0 | 5% (4%; 2%–20%) | 7% (6%; 3%–16%) |
| Mean Self-Citation Rate (Median; Range) | 2% (1%; 0%–14%) | 4% (3%; 1%–13%) | 9% (7%; 3%–22%) |
Subgroup analysis was performed with regard to the academic metrics for each individual subspecialty (Table 3) and the number of positions offered by each program (Table 4). While there was no difference in the change of h-index among any subspecialty, foot and ankle surgeons had a significantly lower total number of citations (1130 vs. 1903, p < 0.05) and h-index (14.5 vs. 18.7, p < 0.04) when compared to the remainder of the cohort. Sports medicine surgeons had a significantly higher total number of citations (2315 vs. 1692, p < 0.05) and h-index (20.9 vs. 17.6, p < 0.04).
Table 3.
Academic metrics by fellowship type.
| # Programs | Mean Publications | p value | Mean Citations | p value | Mean h-index | p value | Self-Citation Rate | p value | |
|---|---|---|---|---|---|---|---|---|---|
| Total | 446 | 71 | 1816 | 18.0 | 3.86% | ||||
| Arthroplasty | 90 | 62 | >0.1 | 1573 | >0.4 | 16.0 | >0.08 | 4.08% | 0.2518 |
| Foot and Ankle | 50 | 55 | >0.1 | 1130 | <0.05 | 14.5 | <0.04 | 2.69% | 0.5972 |
| Oncology | 19 | 88 | >0.3 | 2256 | >0.4 | 23.4 | >0.09 | 6.35% | 0.0676 |
| Trauma | 59 | 64 | >0.4 | 1622 | >0.5 | 17.2 | >0.5 | 2.31% | 0.3880 |
| Pediatrics | 42 | 60 | >0.2 | 1271 | >0.1 | 15.2 | >0.1 | 3.44% | 0.6499 |
| Shoulder and Elbow | 28 | 86 | >0.2 | 2095 | >0.5 | 23.0 | >0.06 | 3.37% | 0.3019 |
| Spine | 69 | 84 | >0.1 | 2162 | >0.2 | 19.9 | >0.2 | 4.29% | 0.6371 |
| Sports | 89 | 81 | >0.1 | 2218 | <0.05 | 20.9 | <0.04 | 4.18% | 0.8581 |
Table 4.
Academic metrics by number of fellows.
| # Programs | Mean Publications | p value | Mean Citations | p value | Mean h-index | p value | Self-Citation Rate | p value | |
|---|---|---|---|---|---|---|---|---|---|
| Total | 446 | 71 | 1816 | 18.0 | 3.86% | ||||
| 1 Fellow | 227 | 51 | <0.0001 | 1141 | <0.0001 | 14.2 | <0.0001 | 3.08% | >0.03 |
| >1 Fellow | 219 | 92 | <0.0001 | 2515 | <0.0001 | 22.3 | <0.0001 | 4.23% | >0.03 |
| >2 Fellows | 101 | 115 | <0.0001 | 3600 | <0.0001 | 27.1 | <0.0001 | 4.74% | <0.0006 |
| >3 Fellows | 50 | 122 | <0.0001 | 3998 | <0.0001 | 29.4 | <0.0001 | 5.53% | <0.0001 |
Directors of programs offering more than one fellowship position had significantly higher number of publications (92 vs. 51, p < 0.0001), total number of citations (2515 vs. 1141, p < 0.0001), h-index (22.3 vs. 14.2, p < 0.0001), and mean self-citation rate (3.08% vs. 4.23%, p < 0.04) than those only offering one position. The change remained significant and was more pronounced with increasing number of fellowship positions. Directors of programs offering more than three positions had a mean number of 122 publications, 3998 citations, h-index of 29.4, and a self-citation rate of 5.53%.
4. Discussion
Nearly 80% of program directors had no change in their h-index with self-citations removed and 95% had a change in one integer or less when excluding self-citations. As would be anticipated, program directors with changes in their h-indices had significantly more publications, overall citations, and higher h-indices when compared to those who were unchanged. Not surprisingly, as the number of fellowship positions increased, there were associated significant increases in number of publications, overall citations, h-index, change in h-index, and self-citation rate. Schoenfield et al. and Khan et al. have published similar results describing the relationship of fellowship size to both number of publications and h-index.7,9
The h-index has been suggested to impact external funding of research. Buerba et al. reported that academic surgeons who receive industry support or NIH funding tend to have higher h-indices than otherwise.11 Using the h-index, Stavrakis et al. reported that academic success of an orthopaedic department is directly associated with scholarly productivity and funding of both the department chairs and the research directors.12 In the current cohort, program directors had a mean h-index of 18.3, while the mean h-index with self-citations removed was 18.0. An h-index of 18.3 is comparable to the number in the study by Stavrakis et al. (mean h-index 20)12, and of National Institutes of Health (NIH) funded orthopaedic surgeons analyzed by Silvestre et al. (mean h-index 24).17 Program directors in this study with a change in h-index had a significantly higher rate of self-citation (5.70% vs. 3.86% for the entire cohort), though these figures fall within the range of both orthopaedic chairpersons (3.96%), and NIH funded orthopaedic surgeons (7.16%).17 We hypothesize that the increased rate of self-citation seen by high volume academic surgeons is secondary to continued topical research which necessarily builds off of previous publication. However, given that self-citation caused an increase in h-index of one integer or less in 94% of the cohort, it would be difficult to conclude that self-citation needs to be adjusted for when determining distribution of external funding or grants for research, or as it pertains to academic positions and promotions.
While sub-specialty did not affect a change in h-index, there were other significant differences observed. Foot and ankle surgery program directors had significantly lower overall citations and h-indices, while sports medicine program directors had significantly higher overall citations and h-indices. Though no studies currently exist examining the h-index of foot and ankle surgeons, our results of sports medicine program directors (mean h-index 20.9) are similar to an examination of sports medicine associate professors and professors by Cvetanovich et al., in 2016. In that study, associate professors and professors specializing in sports medicine had mean h-indices of 14 and 28, respectively.4 Though we cannot conclude why such differences exist among these two specialties, recent studies of orthopaedic journal impact factors may provide some insight. In 2010, the two highest impact factors for orthopaedic surgery journals encompassed sports medicine (American Journal of Sports Medicine and Arthroscopy), while the highest rated dedicated foot and ankle journal ranked 24th (Foot & Ankle International).19 A similar study was performed in 2016 where the American Journal of Sports Medicine retained the top spot, while Foot & Ankle International only improved to 22nd.20 Another potential reason for the differences observed could be that h-index is not a true linear score. For example, it is much more difficult to raise an h-index from 7 to 15, than it is from 15 to 23. Therefore, while there were significant statistical differences observed between sports medicine and foot and ankle surgery, these differences may not be as significant on a clinical/practical level. Despite this, our findings suggest when using the h-index as a measure of productivity, appropriate benchmarks may need to be in place for specific subspecialties, particularly if it translates to payment/salary, funding, or promotional considerations at the departmental level.
Our results demonstrate that self-citation for orthopaedic fellowship program directors - excluding hand surgery - was present in almost 4% of all citations. While self-citation can alter an h-index, it does not necessarily correlate with self-promotion of academic productivity.21 Publication of longitudinal and compounding research projects often requires necessary, and beneficial, self-citation in the reporting and comparison of orthopaedic outcomes through follow-up at the short-, mid-, and long-term ranges.22 Previous studies have demonstrated the prevalence of self-citation in the orthopaedic community and demonstrated that its impact is minimal. Silvestre et al. evaluated the impact of self-citation on orthopaedic investigators from various backgrounds, including program directors, chairpersons, and faculty at various residency programs in the United States.17 They noted an overall self-citation rate of 5.8% from 463 various researchers, with increasing number of publications correlating to a proportional increase in h-index and self-citation – similar to the current study. Approximately 64% of their cohort had no change in their h-index due to self-citation versus 78% in fellowship program directors. Lopez et al. assessed the magnitude of self-citation amongst a cohort of academic hand surgeons using the h-index.23 They identified 354 full-time academic hand surgeons with an average of 45 publications. The average self-citation rate was 2.2%, half of that in the current study of other orthopaedic sub-specialties. Similar to the current study, with increasing publications the self-citation rate also rose. H-index was unchanged in approximately 84% of the surgeons when eliminating self-citation.
The h-index has been used previously as a measure of academic productivity in orthopaedic residency and fellowship training.2, 3, 4, 5, 6, 7, 8, 9, 10 Bastian et al. utilized it, as well as the m- and e-indices to demonstrate academic productivity's correlation to academic rank in orthopaedic residency programs.2 Ence et al. also utilized the h-index and m-index to show that higher values correlated with higher academic orthopaedic faculty rank in 4,663 surgeons at 142 institutions.5 Post et al. used the h-index to evaluate neurosurgical and orthopaedic spine surgery faculty and how they rank based off of academic productivity as well as case volume.8 Similarly, Schoenfeld at al. suggested that the h-index and number of publications could help surgeons benchmark performance of a spine fellowship program.9 Bernstein et al. demonstrated that Clinician Scholar Career Development Program participants with higher h-indices were more likely to be awarded fundings, grants, or traveling fellowships.3 Similar to some of the results found in the present study, Khan et al. identified 375 adult reconstruction surgeons in fellowship training facilities and found that increasing number of fellows, faculty academic title, years in practice, and formal fellowship training all correlated with increased numbers of publication and higher h-indices.7 Cvetanovich et al. identified 610 faculty of fellowships from the American Orthopaedic Society for Sports Medicine. They noted that higher academic program rank correlated with cumulative higher h-index and longer time in practice since fellowship. Similar to the present study, fellowships with a higher number of fellows had more publications per faculty member, higher cumulative h-index, and more American Journal of Sports Medicine and Arthroscopy publications per faculty member.4 None of the previously listed studies, however, accounted for self-citation rate and its potential impact on h-index.
Limitations to this study include that it is an investigation solely of fellowship program directors, and is not necessarily a reflection of orthopaedic fellowship programs as a whole. Program directors may, or may not, be the most academically productive member of their respective departments. Thus, we cannot specifically state that a particular subspecialty is more or less productive than the other. Second, while the h-index can provide a number-value for academic productivity in relation to number of publications and citations, it does not account for the individual impact of different journals, or different types of studies performed.24 Lastly, we analyzed the h-index at a single point in time and did not account for how a researcher's productivity is evolving overtime. It is possible that self-citation is affecting productivity at varying rates depending on the researcher's utilization of such. Future studies would be beneficial in accounting for these changes over time.
Other limitations exist within the SF match website, the Scopus database, and its search engine. There are multiple criticisms concerning the availability and accuracy of information on the SF match website across multiple orthopaedic subspecialties.25, 26, 27, 28, 29 In addition, Scopus search queries can return various results ranging from a single corresponding author (more common with uncommon names), or to a plethora of authors (more common with common names). When numerous authors are returned in a search, a manual inspection of the author's name, institution, and type of publication had to be conducted for complete inclusion leading to the addition of potential human error in the process of the database query. This occurrence was infrequent, however, and we believe such occurrences had minimal impact on the study results. Lastly, the Scopus database is not a “gold standard” for author metrics and previous studies have elucidated poor agreement between Scopus and other databases.30
In conclusion, the rate of self-citation amongst orthopaedic surgery fellowship program directors is approximately 4%. Self-citation has minimal impact on the h-index as a measure of academic productivity as 95% of fellowship program directors demonstrated a change of one integer or less. This suggests the h-index does not need to be adjusted for self-citation when used as a correlative for the distribution of external funding or grants for research. All metrics increased with increasing number of fellowship positions. Additionally, sports medicine surgeons have significantly higher h-indices and total citations than other subspecialties, while foot and ankle surgeons have lower respective metrics, suggesting that appropriate benchmarks may be necessary for varying department sizes and subspecialties as it pertains to payment/salary, funding, or promotional considerations at the departmental level.
Location of research
This research was performed at Blanchfield Army Community Hospital, 650 Joel Dr., Fort Campbell, KY 42223.
Additional note
The views and opinions expressed in this article are those solely of the authors and in no way reflect the opinions of the United States Government.
Declaration of competing interest
Dr. Justin Ernat is a paid consultant for Johnson & Johnson Service, Inc. – Depuy Synthes Mitek Sports Medicine. Neither Dr. Ernat, nor any immediate family member, has received anything of value from or has stock or stock options held in any commercial company or institution related directly or indirectly to the subject of this article.
Dr. Christopher Yheulon and Dr. Andrew Lopez have no conflicts of interest to report.
Dr. Lucian Warth is a paid consultant of Stryker Orthopaedics, LINK Orthopaedics, and Osteoremedies. Neither Dr. Warth, nor any immediate family member, has received anything of value from or has stock or stock options held in any commercial company or institution related directly or indirectly to the subject of this article.
The authors received external no funding, grants, or other support for the creation of this manuscript to include the following organizations: National Institutes of Health (NIH); Wellcome Trust; Howard Hughes Medical Institute (HHMI); and other(s).
Contributor Information
Justin J. Ernat, Email: justin.ernat@gmail.com.
Christopher G. Yheulon, Email: Christopher.yheulon@gmail.com.
Andrew J. Lopez, Email: Andrew.j.lopez30.mil@mail.mil.
Lucian C. Warth, Email: lwarth@uihealth.org.
References
- 1.Hirsch J.E. An index to quantify an individual's scientific research output. Proc. Natl. Acad. Sci. U.S.A. 2005;102(46):16569–16572. doi: 10.1073/pnas.0507655102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bastian S., Ippolito J.A., Lopez S.A., Eloy J.A., Beebe K.S. The use of the h-index in academic orthopaedic surgery. J Bone Jt Surg Am Vol. 2017;99(4):e14. doi: 10.2106/JBJS.15.01354. [DOI] [PubMed] [Google Scholar]
- 3.Bernstein D.N., Lawson M., Menga E.N., O'Keefe R.J., Rubery P.T., Mesfin A. Scholarly success of orthopaedic surgeons participating in the clinician scholar career development program. J Bone Jt Surg Am Vol. 2018;100(17):e115. doi: 10.2106/JBJS.17.01605. [DOI] [PubMed] [Google Scholar]
- 4.Cvetanovich G.L., Saltzman B.M., Chalmers P.N., Frank R.M., Cole B.J., Bach B.R., Jr. Research productivity of sports medicine fellowship faculty. Orthop J Sports Med. 2016;4(12) doi: 10.1177/2325967116679393. 2325967116679393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Ence A.K., Cope S.R., Holliday E.B., Somerson J.S. Publication productivity and experience: factors associated with academic rank among orthopaedic surgery faculty in the United States. J Bone Jt Surg Am Vol. 2016;98(10):e41. doi: 10.2106/JBJS.15.00757. [DOI] [PubMed] [Google Scholar]
- 6.Jones L.B., Goel S., Hung L.Y. Objective methodology to assess meaningful research productivity by orthopaedic residency departments: validation against widely distributed ranking metrics and published surrogates. J Orthop Trauma. 2018;32(4):e139–e144. doi: 10.1097/BOT.0000000000001106. [DOI] [PubMed] [Google Scholar]
- 7.Khan A.Z., Kelley B.V., Patel A.D., McAllister D.R., Leong N.L. Academic productivity among fellowship associated adult total joint reconstruction surgeons. Arthroplasty today. 2017;3(4):298–302. doi: 10.1016/j.artd.2017.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Post A.F., Li A.Y., Dai J.B. Academic productivity of spine surgeons at United States neurological surgery and orthopedic surgery training programs. World neurosurgery. 2019;121:e511–e518. doi: 10.1016/j.wneu.2018.09.150. [DOI] [PubMed] [Google Scholar]
- 9.Schoenfeld A.J., Bhalla A., George J., Harris M.B., Bono C.M. Academic productivity and contributions to the literature among spine surgery fellowship faculty. Spine J : Off J North Am Spine Soc. 2015;15(10):2126–2131. doi: 10.1016/j.spinee.2015.03.026. [DOI] [PubMed] [Google Scholar]
- 10.Wei M., Wang W., Zhuang Y. Worldwide research productivity in the field of spine surgery: a 10-year bibliometric analysis. Eur Spine J : Off Publ Eur Spine Soc Eur Spinal Deformity Soc Eur Sect cervical Spine Res Soc. 2016;25(4):976–982. doi: 10.1007/s00586-016-4442-3. [DOI] [PubMed] [Google Scholar]
- 11.Buerba R.A., Sheppard W.L., Herndon K.E. Academic influence and its relationship to industry payments in orthopaedic surgery. J Bone Jt Surg Am Vol. 2018;100(9):e59. doi: 10.2106/JBJS.17.00838. [DOI] [PubMed] [Google Scholar]
- 12.Stavrakis A.I., Patel A.D., Burke Z.D. The role of chairman and research director in influencing scholarly productivity and research funding in academic orthopaedic surgery. J Orthop Res : Off Publ Orthop Res Soc. 2015;33(10):1407–1411. doi: 10.1002/jor.22919. [DOI] [PubMed] [Google Scholar]
- 13.Rad A.E., Shahgholi L., Kallmes D. Impact of self-citation on the H index in the field of academic radiology. Acad Radiol. 2012;19(4):455–457. doi: 10.1016/j.acra.2011.11.013. [DOI] [PubMed] [Google Scholar]
- 14.Susarla S.M., Swanson E.W., Lopez J., Peacock Z.S., Dodson T.B. Does self-citation influence quantitative measures of research productivity among academic oral and maxillofacial surgeons? J Oral Maxillofac Surg : Off J Am Assoc Oral Maxillofac.Surg. 2015;73(10) doi: 10.1016/j.joms.2015.05.018. 1981.e1981-1987. [DOI] [PubMed] [Google Scholar]
- 15.Lopez J., Susarla S.M., Swanson E.W., Luck J.D., Tuffaha S., Lifchez S.D. The effect of self-citations on the Hirsch index among full-time academic hand surgeons. J Surg Educ. 2016;73(2):317–322. doi: 10.1016/j.jsurg.2015.10.012. [DOI] [PubMed] [Google Scholar]
- 16.Swanson E.W., Miller D.T., Susarla S.M. What effect does self-citation have on bibliometric measures in academic plastic surgery? Ann Plast Surg. 2016;77(3):350–353. doi: 10.1097/SAP.0000000000000585. [DOI] [PubMed] [Google Scholar]
- 17.Silvestre J., Kamath A.F. Prevalence and impact of self-citation in academic orthopedic surgery. Am J Orthoped. 2018;47(3) doi: 10.12788/ajo.2018.0015. [DOI] [PubMed] [Google Scholar]
- 18.Tolisano A.M., Song S.A., Cable B.B. Author self-citation in the otolaryngology literature: a pilot study. Otolaryngology--head and neck surgery. Off J Am Acad Otolaryngology-Head Neck Surg. 2016;154(2):282–286. doi: 10.1177/0194599815616111. [DOI] [PubMed] [Google Scholar]
- 19.Moverley R., Rankin K.S., McNamara I., Davidson D.J., Reed M., Sprowson A.P. Impact factors of orthopaedic journals between 2000 and 2010: trends and comparisons with other surgical specialties. Int Orthop. 2013;37(4):561–567. doi: 10.1007/s00264-012-1769-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Mohamed N.S., Gwam C.U., Etcheson J.I. Impact factors of orthopaedic journals between 2010 and 2016: trends and comparisons with other surgical specialties. Ann Transl Med. 2018;6(7):114. doi: 10.21037/atm.2018.03.02. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bartneck C., Kokkelmans S. Detecting h-index manipulation through self-citation analysis. Scientometrics. 2011;87(1):85–98. doi: 10.1007/s11192-010-0306-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Ioannidis J.P. A generalized view of self-citation: direct, co-author, collaborative, and coercive induced self-citation. J Psychosom Res. 2015;78(1):7–11. doi: 10.1016/j.jpsychores.2014.11.008. [DOI] [PubMed] [Google Scholar]
- 23.Lopez J., Ameri A., Susarla S.M. Does formal research training lead to academic success in plastic surgery? A comprehensive analysis of U.S. Academic plastic surgeons. J Surg Educ. 2016;73(3):422–428. doi: 10.1016/j.jsurg.2015.12.001. [DOI] [PubMed] [Google Scholar]
- 24.Bhandari M., Busse J., Devereaux P.J. Factors associated with citation rates in the orthopedic literature. Can J Surg J Can Chir. 2007;50(2):119–123. [PMC free article] [PubMed] [Google Scholar]
- 25.Mulcahey M.K., Gosselin M.M., Fadale P.D. Evaluation of the content and accessibility of web sites for accredited orthopaedic sports medicine fellowships. J Bone Jt Surg Am Vol. 2013;95(12):e85. doi: 10.2106/JBJS.L.00785. [DOI] [PubMed] [Google Scholar]
- 26.Davidson A.R., Murphy R.F., Spence D.D., Kelly D.M., Warner W.C., Jr., Sawyer J.R. Accessibility and quality of online information for pediatric orthopaedic surgery fellowships. J Pediatr Orthop. 2014;34(8):831–834. doi: 10.1097/BPO.0000000000000217. [DOI] [PubMed] [Google Scholar]
- 27.Silvestre J., Guzman J.Z., Skovrlj B. The Internet as a communication tool for orthopedic spine fellowships in the United States. Spine J : Off J North Am Spine Soc. 2015;15(4):655–661. doi: 10.1016/j.spinee.2014.11.024. [DOI] [PubMed] [Google Scholar]
- 28.Yayac M., Javandal M., Mulcahey M.K. Accredited orthopaedic sports medicine fellowship websites: an updated assessment of accessibility and content. Orthop J Sports Med. 2017;5(1) doi: 10.1177/2325967116683942. 2325967116683942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Shaath M.K., Yeranosian M.G., Ippolito J.A., Adams M.R., Sirkin M.S., Reilly M.C. Evaluation of the content and accessibility of web sites for accredited orthopaedic trauma surgery fellowships. J Bone Jt Surg Am Vol. 2018;100(9):e60. doi: 10.2106/JBJS.17.01112. [DOI] [PubMed] [Google Scholar]
- 30.Walker B., Alavifard S., Roberts S., Lanes A., Ramsay T., Boet S. Inter-rater reliability of h-index scores calculated by Web of Science and Scopus for clinical epidemiology scientists. Health Inf Libr J. 2016;33(2):140–149. doi: 10.1111/hir.12140. [DOI] [PubMed] [Google Scholar]