Abstract
Background:
Several studies have analyzed the most cited articles in shoulder, elbow, pediatrics, and foot and ankle surgery. However, no study has analyzed the quality of the most cited articles in elbow medial ulnar collateral ligament (UCL) surgery.
Purpose:
To (1) identify the top 50 most cited articles related to UCL surgery, (2) determine whether there was a correlation between the top cited articles and level of evidence, and (3) determine whether there was a correlation between study methodological quality and the top cited articles.
Study Design:
Systematic review.
Methods:
Web of Science and Scopus online databases were searched to identify the top 50 cited articles in UCL surgery. Level of evidence, number of times cited, year of publication, name of journal, country of origin, and study type were recorded for each study. Study methodological quality was analyzed for each article with the Modified Coleman Methodology Score (MCMS) and the Methodological Index for Non-randomized Studies (MINORS). Correlation coefficients were calculated.
Results:
The 50 most cited articles were published between 1981 and 2015. The number of citations per article ranged from 20 to 301 (mean ± SD, 71 ± 62 citations). Most articles (92%) were from the United States and were level 3 (16%), level 4 (58%), or unclassified (16%) evidence. There were no articles of level 1 evidence quality. The mean MCMS and MINORS scores were 28.1 ± 13.4 (range, 3-52) and 9.2 ± 3.6 (range, 2-19), respectively. There was no significant correlation between the mean number of citations and level of evidence or quality (rs = –0.01, P = .917), MCMS (rs = 0.09, P = .571), or MINORS (rs = –0.26, P = .089).
Conclusion:
The top 50 cited articles in UCL surgery constitute a low level of evidence and low methodological quality, including no level 1 articles. There was no significant correlation between the mean number of citations and level of evidence or study methodological quality. However, weak correlations were observed for later publication date and improved level of evidence and methodological quality.
Keywords: elbow, baseball, UCL, Tommy John, quality of evidence
The number of times that an article is cited by other authors has been used as a measure for the academic impact of an article in the medical literature.1,8,10,32 Research productivity for authors and the impact factor for journals are calculated with the number of citations associated with each publication.26 Impact factor is defined as follows: the number of journal citations within a given year from the preceding 2 years (eg, 2017 citation of items published in 2015 or 2016) divided by the total number of potentially citable items (ie, all items published by a journal over those preceding 2 years).12
It was recently reported that the impact factor has gone from a measure of a journal’s citation influence to a surrogate that assesses the scholarly value of work published in that journal.21 As such, several new metrics have been created to objectively assign a methodological quality value to journals based on the number of citations of a journal over a given period, including the Immediacy Index, Scopus SCImago Journal Rank, Scopus CiteScore, and Scopus Source Normalized Impact per Paper. This has led to several studies analyzing the top cited articles in their respective fields.4,19,24,25
Improved understanding of musculoskeletal basic science and the development of new surgical implants and techniques have led to rapid increases in publications in the orthopaedic literature.11,22 As such, several authors have attempted to analyze the most cited articles in shoulder, elbow, pediatrics, and foot and ankle surgery.3,5,15,17,23 However, no study has analyzed the quality of the most cited articles in elbow medial ulnar collateral ligament (UCL) surgery.
Such an assessment is important, as injuries to the elbow UCL are on the rise despite prevention strategies.13 These patients often present with medial elbow pain affecting their throwing velocity and accuracy limiting their availability and effectiveness in games.6 In 1946, Waris31 became the first to describe elbow UCL ruptures (in javelin throwers), but UCL reconstruction was not popularized until Jobe performed the first UCL reconstruction in 1974 on pitcher Tommy John.7 Since that time, the frequency of surgical reconstruction has increased dramatically.6
Also, as researchers are increasingly evaluated and compared by their studies’ academic impact, “top 50” or “top 25” lists of articles on a specific topic are of high value to readerships if, upon critical analysis, their methodological quality supports their “top x” ranking.
The purpose of this study was to (1) identify the top 50 most cited articles related to UCL surgery, (2) determine if there was a correlation between the top cited articles and their level of evidence, and (3) determine if there was a correlation between study methodological quality and the top cited articles. We hypothesized that there would be no significant correlation between the top cited articles in UCL repair surgery and level of evidence or study methodological quality.
Methods
The Web of Science (v 5.23.2; Thomson Reuters) and Scopus online databases were searched in March 2017 according to previously described methods without date restrictions.3,17,18,30 The terms “ulnar collateral ligament of the elbow reconstruction,” “UCL reconstruction,” “ulnar collateral ligament of the elbow repair,” and “UCL repair” were individually searched in each database. All articles and all journals were included. Articles were sorted by the number of times cited, from highest to lowest. Each article was evaluated to determine whether it was appropriately related to UCL surgery. The number of citations for each article was then averaged between the 2 search engines to create a list of the top 50 cited articles in UCL surgery.
Characteristics from each article were recorded: number of times cited, year of publication, name of journal, country of origin, and study type (narrative review, technique guide, animal studies, cadaveric studies, retrospective or prospective case series, cohort investigations, case-control, and randomized controlled trial). Level of evidence for each study was evaluated per the guidelines of the Journal of Bone and Joint Surgery (American).20 Study methodological quality was analyzed for each article with the Modified Coleman Methodology Score (MCMS) and Methodological Index for Non-randomized Studies (MINORS).9,16,27,28
Data were tested for normal distribution with the Kolmogorov-Smirnov test. When the data were normally distributed, the Pearson correlation coefficient (r) was used to determine the correlation between the top cited articles (by mean number of citations) and level of evidence and the correlation between study methodological quality and the top cited articles. When the data were not normally distributed, the Spearman correlation coefficient (rs) was used to determine the degree of correlation between the top cited articles (by mean number of citations) and level of evidence and the degree of correlation between study quality and the top cited articles. Correlation (r and rs) was defined as follows: as a perfect (–1), strong (–0.70), moderate (–0.50), or weak negative linear relationship (–0.30); as no linear relationship (0); or as a weak (0.30), moderate (0.50), strong (0.70), or perfect positive linear relationship (1). Student t tests were carried out for 2 group comparisons. P value <.05 was defined as significant.
Results
The 50 most cited articles in elbow UCL treatment were published between 1981 and 2015 (Table 1). The decade from 2000 to 2009 accounted for the greatest number of articles (n = 23) (Figure 1). The selected articles were published in 12 journals (Figure 2). Most of the articles (n = 29, 58%) were published in the American Journal of Sports Medicine. The mean number of citations ranged from 20 to 301 (mean ± SD, 71 ± 62 citations) (Table 1). Four countries were represented, with 46 (92%) of the top 50 cited articles being from the United States (Figure 3).
TABLE 1.
Top 50 Most Cited Articles in Ulnar Collateral Ligament Surgerya
| Rank | Article | Country | Type | Mean Citationsb | Level of Evidence | MCMS | MINORS |
|---|---|---|---|---|---|---|---|
| 1 | Conway JE, Jobe FW, Glousman RE, Pink M. Medial instability of the elbow in throwing athletes: treatment by repair or reconstruction of the ulnar collateral ligament. J Bone Joint Surg Am. 1992;74(1):67-83. | USA | Case series | 300.5 | 4 | 51 | 9 |
| 2 | Jobe FW, Stark H, Lombardo SJ. Reconstruction of the ulnar collateral ligament in athletes. J Bone Joint Surg Am. 1986;68(8):1158-1163. | USA | Case series | 296 | 4 | 28 | 12 |
| 3 | Cain EL Jr, Dugas JR, Wolf RS, Andrews JR. Elbow injuries in throwing athletes: a current concepts review. Am J Sports Med. 2003;31(4):621-635. | USA | Review | 175.5 | 4 | 12 | 2 |
| 4 | Azar FM, Andrews JR, Wilk KE, Groh D. Operative treatment of ulnar collateral ligament injuries of the elbow in athletes. Am J Sports Med. 2000;28(1):16-23. | USA | Case series | 173.5 | 4 | 28 | 7 |
| 5 | Rohrbough JT, Altchek DW, Hyman J, Williams RJ 3rd, Botts JD. Medial collateral ligament reconstruction of the elbow using the docking technique. Am J Sports Med. 2002;30(4):541-548. | USA | Case series | 150 | 4 | 36 | 11 |
| 6 | Andrews JR, Timmerman LA. Outcome of elbow surgery in professional baseball players. Am J Sports Med. 1995;23(4):407-413. | USA | Case series | 148.5 | 4 | 32 | 8 |
| 7 | Thompson WH, Jobe FW, Yocum LA, Pink MM. Ulnar collateral ligament reconstruction in athletes: muscle-splitting approach without transposition of the ulnar nerve. J Shoulder Elbow Surg. 2001;10(2):152-157. | USA | Case series | 131.5 | 4 | 40 | 7 |
| 8 | Petty DH, Andrews JR, Fleisig GS, Cain EL. Ulnar collateral ligament reconstruction in high school baseball players: clinical results and injury risk factors. Am J Sports Med. 2004;32(5):1158-1164. | USA | Case series | 121 | 3 | 28 | 9 |
| 9 | Cain EL Jr, Andrews JR, Dugas JR, et al. Outcome of ulnar collateral ligament reconstruction of the elbow in 1281 athletes: results in 743 athletes with minimum 2-year follow-up. Am J Sports Med. 2010;38(12):2426-2434. | USA | Case series | 102 | 4 | 38 | 7 |
| 10 | Mirowitz SA, London SL. Ulnar collateral ligament injury in baseball pitchers: MR imaging evaluation. Radiology. 1992;185(2):573-576. | USA | Case series | 99.5 | 2 | 9 | 4 |
| 11 | Smith GR, Altchek DW, Pagnani MJ, Keeley JR. A muscle-splitting approach to the ulnar collateral ligament of the elbow: neuroanatomy and operative technique. Am J Sports Med. 1996;24(5):575-580. | USA | Cadaveric study | 94.5 | NA | 6 | 9 |
| 12 | Ahmad CS, Lee TQ, ElAttrache NS. Biomechanical evaluation of a new ulnar collateral ligament reconstruction technique with interference screw fixation. Am J Sports Med. 2003;31(3):332-337. | USA | Cadaveric study | 93 | NA | 18 | 9 |
| 13 | Dodson CC, Thomas A, Dines JS, Nho SJ, Williams RJ 3rd, Altchek DW. Medial ulnar collateral ligament reconstruction of the elbow in throwing athletes. Am J Sports Med. 2006;34(12):1926-1932. | USA | Case series | 90.5 | 4 | 44 | 13 |
| 14 | Fleisig GS, Andrews JR, Cutter GR, et al. Risk of serious injury for young baseball pitchers: a 10-year prospective study. Am J Sports Med. 2011;39(2):253-257. | USA | Case series | 90 | 3 | 40 | 12 |
| 15 | Vitale MA, Ahmad CS. The outcome of elbow ulnar collateral ligament reconstruction in overhead athletes: a systematic review. Am J Sports Med. 2008;36(6):1193-1205. | USA | Systematic review | 89 | 3 | 30 | 8 |
| 16 | Dines JS, ElAttrache NS, Conway JE, Smith W, Ahmad CS. Clinical outcomes of the DANE TJ technique to treat ulnar collateral ligament insufficiency of the elbow. Am J Sports Med. 2007;35(12):2039-2044. | USA | Case series | 61.5 | 4 | 29 | 9 |
| 17 | Paletta GA Jr, Wright RW. The modified docking procedure for elbow ulnar collateral ligament reconstruction: 2-year follow-up in elite throwers. Am J Sports Med. 2006;34(10):1594-1598. | USA | Case series | 60.5 | 4 | 36 | 11 |
| 18 | Koh JL, Schafer MF, Keuter G, Hsu JE. Ulnar collateral ligament reconstruction in elite throwing athletes. Arthroscopy. 2006;22(11):1187-1191. | USA | Case series | 60 | 4 | 26 | 7 |
| 19 | Field LD, Savoie FH. Common elbow injuries in sport. Sports Med. 1998;26(3):193-205. | USA | Review | 60 | 4 | 10 | 2 |
| 20 | Armstrong AD, Dunning CE, Ferreira LM, Faber KJ, Johnson JA, King GJ. A biomechanical comparison of four reconstruction techniques for the medial collateral ligament–deficient elbow. J Shoulder Elbow Surg. 2005;14(2):207-215. | Canada | Cadaveric study | 60 | NA | 24 | 13 |
| 21 | Norwood LA, Shook JA, Andrews JR. Acute medial elbow ruptures. Am J Sports Med. 1981;9(1):16-19. | USA | Case series | 59.5 | 4 | 20 | 11 |
| 22 | O’Driscoll SW. Elbow instability. Hand Clin. 1994;10(3):405-415. | USA | Review | 57.5 | 4 | 0 | 0 |
| 23 | Erickson BJ, Gupta AK, Harris JD, et al. Rate of return to pitching and performance after Tommy John surgery in Major League Baseball pitchers. Am J Sports Med. 2014;42(3):536-543. | USA | Case series | 56.5 | 3 | 52 | 10 |
| 24 | Hechtman KS, Tjin-A-Tsoi EW, Zvijac JE, Uribe JW, Latta LL. Biomechanics of a less invasive procedure for reconstruction of the ulnar collateral ligament of the elbow. Am J Sports Med. 1998;26(5):620-624. | USA | Cadaveric study | 46.5 | NA | 21 | 12 |
| 25 | Savoie FH 3rd, Trenhaile SW, Roberts J, Field LD, Ramsey JR. Primary repair of ulnar collateral ligament injuries of the elbow in young athletes: a case series of injuries to the proximal and distal ends of the ligament. Am J Sports Med. 2008;36(6):1066-1072. | USA | Case series | 42 | 4 | 41 | 11 |
| 26 | Bennett JB, Green MS, Tullos HS. Surgical management of chronic medial elbow instability. Clin Orthop Relat Res. 1992;(278):62-68. | USA | Case series | 40.5 | 4 | 33 | 10 |
| 27 | Gibson BW, Webner D, Huffman GR, Sennett BJ. Ulnar collateral ligament reconstruction in Major League Baseball pitchers. Am J Sports Med. 2007;35(4):575-581. | USA | Case series | 40.5 | 2 | 48 | 12 |
| 28 | Makhni EC, Lee RW, Morrow ZS, Gualtieri AP, Gorroochurn P, Ahmad CS. Performance, return to competition, and reinjury after Tommy John Surgery in Major League Baseball pitchers: a review of 147 cases. Am J Sports Med. 2014;42(6):1323-1332. | USA | Case series | 37.5 | 3 | 49 | 10 |
| 29 | Bowers AL, Dines JS, Dines DM, Altchek DW. Elbow medial ulnar collateral ligament reconstruction: clinical relevance and the docking technique. J Shoulder Elbow Surg. 2010;19(2)(suppl):110-117. | USA | Case series | 35 | 4 | 30 | 10 |
| 30 | Podesta L, Crow SA, Volkmer D, Bert T, Yocum LA. Treatment of partial ulnar collateral ligament tears in the elbow with platelet-rich plasma. Am J Sports Med. 2013;41(7):1689-1694. | USA | Case series | 34 | 4 | 34 | 10 |
| 31 | Paletta GA Jr, Klepps SJ, Difelice GS, et al. Biomechanical evaluation of 2 techniques for ulnar collateral ligament reconstruction of the elbow. Am J Sports Med. 2006;34(10):1599-1603. | USA | Cadaveric study | 34 | NA | 33 | 13 |
| 32 | Armstrong AD, Dunning CE, Faber KJ, Johnson JA, King GJ. Single-strand ligament reconstruction of the medial collateral ligament restores valgus elbow stability. J Shoulder Elbow Surg. 2002;11(1):65-71. | Canada | Cadaveric study | 32 | NA | 15 | 7 |
| 33 | Hyman J, Breazeale NM, Altchek DW. Valgus instability of the elbow in athletes. Clin Sports Med. 2001;20(1):25-45. | USA | Systematic review | 31 | 5 | 10 | 2 |
| 34 | Lee ML, Rosenwasser MP. Chronic elbow instability. Orthop Clin North Am. 1999;30(1):81-89. | USA | Systematic review | 30 | 5 | 6 | 2 |
| 35 | Eygendaal D. Ligamentous reconstruction around the elbow using triceps tendon. Acta Orthop Scand. 2004;75(5):516-523. | Netherlands | Case series | 29.5 | 4 | 3 | 4 |
| 36 | Bushnell BD, Anz AW, Noonan TJ, Torry MR, Hawkins RJ. Association of maximum pitch velocity and elbow injury in professional baseball pitchers. Am J Sports Med. 2010;38(4):728-732. | USA | Prospective cohort | 28.5 | 3 | 36 | 12 |
| 37 | Dines JS, Yocum LA, Frank JB, ElAttrache NS, Gambardella RA, Jobe FW. Revision surgery for failed elbow medial collateral ligament reconstruction. Am J Sports Med. 2008;36(6):1061-1065. | USA | Case series | 27 | 4 | 26 | 11 |
| 38 | Large TM, Coley ER, Peindl RD, Fleischli JE. A biomechanical comparison of 2 ulnar collateral ligament reconstruction techniques. Arthroscopy. 2007;23(2):141-150. | USA | Cadaveric study | 24 | NA | 24 | 14 |
| 39 | Hariri S, Safran MR. Ulnar collateral ligament injury in the overhead athlete. Clin Sports Med. 2010;29(4):619-644. | USA | Systematic review | 23 | 5 | 10 | 4 |
| 40 | Jiang JJ, Leland JM. Analysis of pitching velocity in Major League Baseball players before and after ulnar collateral ligament reconstruction. Am J Sports Med. 2014;42(4):880-885. | USA | Case-control | 21.5 | 3 | 57 | 19 |
| 41 | Dines JS, Jones KJ, Kahlenberg C, Rosenbaum A, Osbahr DC, Altchek DW. Elbow ulnar collateral ligament reconstruction in javelin throwers at a minimum 2-year follow-up. Am J Sports Med. 2012;40(1):148-151. | USA | Case series | 21.5 | 4 | 29 | 9 |
| 42 | Jones KJ, Conte S, Patterson N, ElAttrache NS, Dines JS. Functional outcomes following revision ulnar collateral ligament reconstruction in Major League Baseball pitchers. J Shoulder Elbow Surg. 2013;22(5):642-646. | USA | Case series | 21 | 4 | 21 | 10 |
| 43 | Marshall NE, Keller RA, Lynch JR, Bey MJ, Moutzouros V. Pitching performance and longevity after revision ulnar collateral ligament reconstruction in Major League Baseball pitchers. Am J Sports Med. 2015;43(5):1051-1056. | USA | Case series | 20.5 | 3 | 43 | 12 |
| 44 | McAdams TR, Lee AT, Centeno J, Giori NJ, Lindsey DP. Two ulnar collateral ligament reconstruction methods: the docking technique versus bioabsorbable interference screw fixation—a biomechanical evaluation with cyclic loading. J Shoulder Elbow Surg. 2007;16:224-228. | USA | Cadaveric study | 20 | NA | 24 | 13 |
| 45 | Cohen SB, Sheridan S, Ciccotti MG. Return to sports for professional baseball players after surgery of the shoulder or elbow. Sports Health. 2011;3(1):105-111. | USA | Case series | 20 | 4 | 34 | 12 |
| 46 | Jones KJ, Dines JS, Rebolledo BJ, et al. Operative management of ulnar collateral ligament insufficiency in adolescent athletes. Am J Sports Med. 2014;42(1):117-121. | USA | Case series | 18.5 | 4 | 32 | 11 |
| 47 | Richard MJ, Aldridge JM, Wiesler ER, Ruch DS. Traumatic valgus instability of the elbow: pathoanatomy and results of direct repair. J Bone Joint Surg Am. 2008;90:2416-2422. | USA | Case series | 18 | 4 | 38 | 12 |
| 48 | Argo D, Trenhaile SW, Savoie FH, Field LD. Operative treatment of ulnar collateral ligament insufficiency of the elbow in female athletes. Am J Sports Med. 2006;34(3):431-437. | USA | Case series | 17 | 4 | 24 | 8 |
| 49 | Hechtman KS, Svijac JE, Wells ME, Botto-van Bemden A. Long-term results of ulnar collateral ligament reconstruction in throwing athletes based on a hybrid technique. Am J Sports Med. 2011;39(2):342-347. | USA | Case series | 17 | 4 | 27 | 9 |
| 50 | Kim BS, Park KH, Song HS, Park, SY. Ligamentous repair of acute lateral collateral ligament rupture of the elbow. J Shoulder Elbow Surg. 2013;22:1469-1473. | South Korea | Case series | 16.5 | 4 | 19 | 10 |
aMCMS, Modified Coleman Methodology Score; MINORS, Methodological Index for Non-randomized Studies; NA, not applicable.
bNumber of citations for each article averaged between the 2 search engines.
Figure 1.
Number of most cited articles and decade of publication.
Figure 2.
Number of top 50 cited articles in each journal. AJSM, American Journal of Sports Medicine; CORR, Clinical Orthopaedics and Related Research; JBJS-Am, Journal of Bone and Joint Surgery (American); JSES, Journal of Shoulder and Elbow Surgery.
Figure 3.
Countries represented in top 50 most cited articles.
The most common type of article was retrospective case series, with 33 (66%) (Figure 4). The studies had evidence levels ranging from 2 to 5 and unclassified (eg, cadaveric studies), with the most common being level 4 (Figure 5). There was no significant correlation between the mean number of citations and level of evidence (rs = –0.01, P = .917) among the studies included in the review.
Figure 4.
Top 50 cited articles by article type. Review, narrative review.
Figure 5.
Top 50 cited articles by level of evidence.
The mean MCMS was 28.1 ± 13.4 (poor; range, 3-52). There was no significant correlation between the mean number of citations and MCMS (rs = 0.09, P = .571). The mean MINORS was 9.2 ± 3.6 (range, 2-19). There was no significant correlation between the mean number of citations and MINORS (rs = –0.26, P = .089).
Studies were analyzed for change over time. There was a weak negative correlation between year published and level of evidence (rs = –0.35, P = .037). Weak positive correlations occurred between year published and MCMS (r = 0.48, P = .001) and between year published and MINORS (rs = 0.35, P = .020).
Discussion
This study identified the top 50 most cited articles related to UCL treatment. There was no significant correlation between the number of citations for the top 50 cited articles and level of evidence or study methodological quality. This study confirmed all our hypotheses. Most studies (80%) had low levels of evidence (levels 4 or 5 or unclassified) and low methodological quality (poor MCMS, low MINORS). However, with time, weak correlations were observed for later publication date and improved level of evidence and methodological quality.
One previous study evaluated the top cited articles in elbow surgery. Huo et al15 demonstrated that among the top 50 articles cited in elbow surgery, the majority were published in the 1990s (n = 18) and 1980s (n = 19). These results intuitively make sense, given that time since publication would increase the likelihood of citation. Our results differ in that the majority of cited articles were published after the year 2000, although our 2 most cited papers were among the oldest included in the study. However, these findings may be related to an increase in the overall number of publications over the past decade inflating and saturating the databases.11,22 Conversely, it may reflect an increase in UCL surgery, as such injuries continue to rise.13
In the current study, the majority of articles were published in the American Journal of Sports Medicine (58%) and the Journal of Shoulder and Elbow Surgery (14%). This differs from a prior study assessing elbow surgery, with only 12% and 0% of the top cited articles coming from the American Journal of Sports Medicine and Journal of Shoulder and Elbow Surgery, respectively.15 Similar to the current study, the Journal of Bone and Joint Surgery (American) was a major source of top cited articles. This is to be expected, as these 3 journals consistently have the highest impact factors in the orthopaedic surgery literature.29
Most authors (92%) from the top 50 cited articles were from the United States. This trend was seen in previous citation studies of shoulder surgery, plastic surgery, general surgery, and orthopaedic surgery (top 100 articles).17–19,23,25 This finding indicates a possible bias toward American authors, as several of the top journals in orthopaedic surgery are based in the United States and published in the English language. However, it is also possible that the majority of the UCL articles are from the United States because baseball is predominately a US sport.
Previous studies showed that the majority of the top cited articles in orthopaedic and elbow surgery are level 4 case series.17,18,23 This was similar to the present study on UCL surgery, in which 25 (50%) articles were level 4 case series. With the recent focus on evidence-based medicine, this study found weak correlations for later publication date and improved level of evidence and methodological quality. However, low-level studies still dominate the literature and should serve as an impetus to improve the methodological quality of studies investigating this topic.
It is possible that this new emphasis is not yet reflected in the number of citations and that if this study were repeated at a later date, the number of investigations with low levels of evidence would decrease. This trend is apparent in a comparison of a study by Namdari et al23 (published in 2012; level 1 studies, n = 0) with a more recent study by Kraeutler et al17 (published in 2016; level 1 studies, n = 3).18 In addition, analogous to the calculation of impact factor, more recent citation data—2 or 5 years rather than the past 30 years (as used in the current study)—would be highly likely to change the composition of the current investigation’s top 50 cited list.
In contrast to previous papers, the current study analyzed the methodological quality of the top cited articles. According to the MCMS, all articles were poor (scores <55), with the overall mean MCMS classified as poor (28.1). Harris et al14 found similar results in a topic not investigated in other top citation articles—specifically, poor overall methodological quality in articular cartilage studies. The methodological quality deficiencies identified in this study should guide the future study design, conduct, and reporting of UCL surgery.
Additionally, the current study aimed to determine if there was any correlation between level of evidence and study methodological quality with number of citations. As expected, there was no significant correlation between level of evidence and number of citations. This is in contrast to a study by Arshi et al,2 which evaluated the top cited articles in cartilage surgery for which the number of citations was correlated with a stronger level of evidence. However, there was a higher number of level 1 and 2 studies in that study compared with the present study.
There was also no significant correlation between the mean number of citations and MCMS and MINORS. The poor correlation between study quality and number of citations is due to the small number of level 1 and 2 studies as compared with the lower-level studies. Additionally, the poor correlation between number of citations and methodological quality is likely secondary to the MCMS favoring randomized controlled trials, while the MINORS favors nonrandomized controlled trials. However, study methodological quality appears to be improving with time, as there was a weak correlation between year published and MCMS and MINORS in the present study. The “classic” articles are typically going to be case studies and therefore have lower levels of evidence, since they initially present the injury and surgical repair technique. Additionally, UCL injuries are still relatively uncommon as compared with other sport-related injuries, making it more challenging to perform level 1 or 2 studies.
This study has a number of strengths and limitations. It was the first to analyze the most cited articles in elbow UCL surgery. The number of articles (N = 50) was arbitrarily chosen and may have eliminated other influential articles. However, this number is based on several previously published studies.3,15,17,23 This study also utilized only 2 databases: Web of Science and Scopus. However, Scopus is the world’s largest scientific database, and by combining it with Web of Science, we believe that few to no citations were missed. Additionally, the search in this study was not limited to “known” orthopaedic journals; rather, it extended to previously published papers to limit the potential for missed relevant articles.
The number of citations can be influenced by a number of factors. One such potential is high-volume authors citing their own work (“self-citation”), which was not accounted for in this study. Also there appears to be a “snowball effect” to citations, as other authors are more likely to cite an article because of previous citations rather than for its content or quality.18 There is also a disadvantage to newer published articles having less time to accrue citations, although this did not seem to be a factor here. Using a 2- or 5-year selection eligibility period (or any arbitrarily chosen time) could change the composition of the top 50 studies in the current investigation’s list.
To objectively analyze the most cited papers, this study included review articles, which served an important purpose: to corroborate the heavily cited low level of evidence. Although review articles often “double dip” certain articles (ie, ones already cited), the fact that they are more heavily cited than other primary articles further demonstrates the overall weakness of the UCL literature. A final limitation is that the number of times an article is cited is always changing and depends not on the quality or content of the article but rather on shifts in the field that change over time.
Conclusion
The top 50 cited articles in UCL surgery constitute a low level of evidence and low methodological quality, including no level 1 articles. There was no significant correlation between the mean number of citations and level of evidence or study methodological quality. However, weak correlations were observed for later publication date and improved level of evidence and methodological quality.
Footnotes
One or more of the authors has declared the following potential conflict of interest or source of funding: P.C.M. has received research support from Arthrex and DePuy; is a paid speaker for Aastrom Biosciences and Vericel; and has received financial/material support for a sports medicine fellowship from 2Aastrom Biosciences, Arthrex, and MedInc. D.M.L. has received educational support and financial/material support for a sports medicine fellowship from Arthrex and MedInc. J.D.H. has received research support from DePuy and Smith & Nephew; has received educational support from Arthrex and MedInc; is a paid consultant for NIA Magellan and Smith & Nephew; is a paid speaker for Ossur and Smith & Nephew; receives publishing royalties from SLACK Inc; and receives financial/material support for a sports medicine fellowship from Arthrex and MedInc.
References
- 1. Adams AB, Simonson D. Publication, citations, and impact factors of leading investigators in critical care medicine. Respir Care. 2004;49(3):276–281. [PubMed] [Google Scholar]
- 2. Arshi A, Siesener NJ, McAllister DR, Williams RJ, 3rd, Sherman SL, Jones KJ. The 50 most cited articles in orthopedic cartilage surgery. Cartilage. 2016;7(3):238–247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Baldwin KD, Kovatch K, Namdari S, Sankar W, Flynn JM, Dormans JP. The 50 most cited articles in pediatric orthopedic surgery. J Pediatr Orthop B. 2012;21(5):463–468. [DOI] [PubMed] [Google Scholar]
- 4. Baltussen A, Kindler CH. Citation classics in anesthetic journals. Anesth Analg. 2004;98(2):443–451. [DOI] [PubMed] [Google Scholar]
- 5. Bayley M, Brooks F, Tong A, Hariharan K. The 100 most cited papers in foot and ankle surgery. Foot (Edinb). 2014;24(1):11–16. [DOI] [PubMed] [Google Scholar]
- 6. Bruce JR, Andrews JR. Ulnar collateral ligament injuries in the throwing athlete. J Am Acad Orthop Surg. 2014;22(5):315–325. [DOI] [PubMed] [Google Scholar]
- 7. Cain EL, Jr, Andrews JR, Dugas JR, et al. Outcome of ulnar collateral ligament reconstruction of the elbow in 1281 athletes: results in 743 athletes with minimum 2-year follow-up. Am J Sports Med. 2010;38(12):2426–2434. [DOI] [PubMed] [Google Scholar]
- 8. Cheek J, Garnham B, Quan J. What’s in a number? Issues in providing evidence of impact and quality of research(ers). Qual Health Res. 2006;16(3):423–435. [DOI] [PubMed] [Google Scholar]
- 9. Coleman BD, Khan KM, Maffulli N, Cook JL, Wark JD; Victorian Institute of Sport Tendon Study Group. Studies of surgical outcome after patellar tendinopathy: clinical significance of methodological deficiencies and guidelines for future studies. Scand J Med Sci Sports. 2000;10(1):2–11. [DOI] [PubMed] [Google Scholar]
- 10. Evidence-Based Medicine Working Group. Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA. 1992;268(17):2420–2425. [DOI] [PubMed] [Google Scholar]
- 11. Farr J, Cole B, Dhawan A, Kercher J, Sherman S. Clinical cartilage restoration: evolution and overview. Clin Orthop Relat Res. 2011;469(10):2696–2705. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Fersht A. The most influential journals: Impact Factor and Eigenfactor. Proc Natl Acad Sci U S A. 2009;106(17):6883–6884. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Fleisig GS, Andrews JR. Prevention of elbow injuries in youth baseball pitchers. Sports Health. 2012;4(5):419–424. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Harris JD, Erickson BJ, Abrams GD, et al. Methodologic quality of knee articular cartilage studies. Arthroscopy. 2013;29(7):1243–1252 , e5. [DOI] [PubMed] [Google Scholar]
- 15. Huo YQ, Pan XH, Li QB, et al. Fifty top-cited classic papers in orthopedic elbow surgery: a bibliometric analysis. Int J Surg. 2015;18:28–33. [DOI] [PubMed] [Google Scholar]
- 16. Jadad AR, Enkin M, Jadad AR. Randomized Controlled Trials: Questions, Answers, and Musings. 2nd ed Malden, MA: Blackwell Publishing; 2007. [Google Scholar]
- 17. Kraeutler MJ, Freedman KB, MacLeod RA, Schrock JB, Tjoumakaris FP, McCarty EC. The 50 most cited articles in rotator cuff repair research. Orthopedics. 2016;39(6):e1045–e1051. [DOI] [PubMed] [Google Scholar]
- 18. Lefaivre KA, Shadgan B, O’Brien PJ. 100 most cited articles in orthopaedic surgery. Clin Orthop Relat Res. 2011;469(5):1487–1497. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Loonen MP, Hage JJ, Kon M. Plastic surgery classics: characteristics of 50 top-cited articles in four plastic surgery journals since 1946. Plast Reconstr Surg. 2008;121(5):320e–327e. [DOI] [PubMed] [Google Scholar]
- 20. Marx RG, Wilson SM, Swiontkowski MF. Updating the assignment of levels of evidence. J Bone Joint Surg Am. 2015;97(1):1–2. [DOI] [PubMed] [Google Scholar]
- 21. McVeigh ME, Mann SJ. The journal impact factor denominator: defining citable (counted) items. JAMA. 2009;302(10):1107–1109. [DOI] [PubMed] [Google Scholar]
- 22. Moran CJ, Shannon FJ, Barry FP, O’Byrne JM, O’Brien T, Curtin W. Translation of science to surgery: linking emerging concepts in biological cartilage repair to surgical intervention. J Bone Joint Surg Br. 2010;92(9):1195–1202. [DOI] [PubMed] [Google Scholar]
- 23. Namdari S, Baldwin K, Kovatch K, Huffman GR, Glaser D. Fifty most cited articles in orthopedic shoulder surgery. J Shoulder Elbow Surg. 2012;21(12):1796–1802. [DOI] [PubMed] [Google Scholar]
- 24. Ollerton JE, Sugrue M. Citation classics in trauma. J Trauma. 2005;58(2):364–369. [DOI] [PubMed] [Google Scholar]
- 25. Paladugu R, Schein M, Gardezi S, Wise L. One hundred citation classics in general surgical journals. World J Surg. 2002;26(9):1099–1105. [DOI] [PubMed] [Google Scholar]
- 26. Pan RK, Fortunato S. Author Impact Factor: tracking the dynamics of individual scientific impact. Sci Rep. 2014;4:4880. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological Index for Non-randomized Studies (MINORS): development and validation of a new instrument. ANZ J Surg. 2003;73(9):712–716. [DOI] [PubMed] [Google Scholar]
- 28. Tallon C, Coleman BD, Khan KM, Maffulli N. Outcome of surgery for chronic Achilles tendinopathy: a critical review. Am J Sports Med. 2001;29(3):315–320. [DOI] [PubMed] [Google Scholar]
- 29. Thomson Reuters. 2016 Journal Citation Reports Science Edition. Toronto, Canada: Thomson Reuters; 2017. [Google Scholar]
- 30. Virk SS, Yu E. The top 50 articles on minimally invasive spine surgery. Spine (Phila Pa 1976). 2017;42(7):513–519. [DOI] [PubMed] [Google Scholar]
- 31. Waris W. Elbow injuries of javelin-throwers. Acta Chir Scand. 1946;93(6):563–575. [PubMed] [Google Scholar]
- 32. Wears RL, Weber E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002;287(21):2847. [DOI] [PubMed] [Google Scholar]