Abstract
Purpose
To use the top 100 articles pertaining to the shoulder labrum to understand the impact that social medial platforms have on the dissemination of shoulder research and to highlight bibliometric factors associated with Altmetric Attention Scores (AAS) to offer insight into the impact that social media platforms have on the dissemination, attention, and citation of shoulder research publications.
Methods
This was a cross-sectional study. In January 2023, the Altmetric database was searched using the PubMed Medical Subject Headings terms “shoulder labrum.” Articles with the greatest AAS were screened to exclude other topics unrelated to the labrum of the shoulder. The top 100 articles that met inclusion criteria were used in the final analysis. Bibliometric factors pertaining to each study were collected for further analysis of article characteristics in accordance with previous studies.
Results
The Altmetric Database query yielded 619 studies. The top 100 articles with highest AAS were identified, mean Attestation Score was 24.85 ± 55.51, with a range of 7 to 460. The included articles represented 35 journals, with 57 articles attributed to 3 journals: American Journal of Sports Medicine (AJSM; 29%), Arthroscopy: The Journal of Arthroscopic & Related Surgery (Arthroscopy; 19%), and the Journal of Shoulder and Elbow Surgery (JSES; 9%). There was a significant increase in AAS for every decrease in the numerical Level of Evidence value for a study (P = .011) but no association between score and citation rate (P > 005).
Conclusions
Top articles on the shoulder labrum, as defined by high AAS score, are most commonly original clinical research published in 1 of 3 sports medicine journals and performed in the United States or Europe. A decreased numerical Level of Evidence is associated with an increase in AAS score, but there is no association between AAS score and citation rate.
Clinical Relevance
The increasing amount of science and health information shared freely through open-access journals, online servers, and numerous social media channels makes it difficult to measure the impact of research. Using measures such as the Altmetric Attention Score, in isolation or addition to measures of researcher or journal impact, has the potential to provide comprehensive information about the impact of research in the modern world.
The perceived impact of orthopaedic research and academicians is traditionally based on author-level metrics such as number of publications, Hirsch index (h-index),1 and i-10 index,2 as well as journal-level metrics such as impact factor (IF). Such metrics are intended to quantify a researchers’ productivity, which can be used as objective measures for determination of research funding and academic positions.3,4 However, these measures primarily emphasize citation rate within the scientific community and do not account for research attention throughout the much broader general public. With the increasing popularity of open-access journals coupled with reliance on social media for health information sharing,5,6 traditional means of assessing research impact may fail to represent the true impact of a scientific work by medical researchers and health care practitioners. As such, medical researchers and health care practitioners are unable to fully grasp the influence of a publication by exclusively evaluating a work’s bibliometrics.
The Altmetric Attention Score (AAS) is derived from a weighted algorithm based on the volume of social media attention of a given research work and was developed to better represent the distribution and viewership of research by accounting for social media and other nontraditional means of distribution.7,8 The score considers attention of research publications on sources such as news outlets, blogs, LinkedIn, Twitter, and YouTube, each with differing importance in the AAS, and has been associated with citation rates across medical specialties.9,10
The AAS has been used to identify the most impactful articles in sports medicine such as the anterior cruciate ligament (ACL),11 the rotator cuff,12 and the medial ulnar collateral ligament (MUCL).13 The purposes of this study were to use the top 100 articles pertaining to the shoulder labrum to understand the impact that social medial platforms have on the dissemination of shoulder research and to highlight bibliometric factors associated with AAS scores to offer insight into the impact that social media platforms have on the dissemination, attention, and citation of shoulder research publications. We hypothesized that the majority of the top 100 articles about the shoulder labrum would be original clinical research from the United States, would be published in a small group of journals, and that Level of Evidence (LOE) would be associated with the AAS.
Methods
Article Selection Criteria
On January 24, 2023, the Altmetric Explorer database (Digital Science; Holtzbrinck Publishing, Stuttgart, Germany) was searched for articles pertaining to the labrum of the shoulder. The PubMed Medical Subject Headings term “shoulder labrum” was entered into the Altmetric Explorer. After stratification by descending AAS, title and abstract review was performed, with articles unrelated or not focused on the labrum of the shoulder being excluded. The 100 articles pertaining to the labrum of the shoulder with the greatest AAS scores were included for analysis. The overall AAS score, calculated and updated in real time, is determined by averaging relatively weighted social media activity scores that an article receives as well as the number of cumulative citations an article receives (Table 1). The AAS includes activity from multiple online platforms, including Facebook, Twitter, and LinkedIn. There is no limit to the AAS an article receives, as a study can continue to accumulate mentions over time. Articles that have received no online attention or mentions obtain a score of 0. All Altmetric variables from Table 1 were considered in the analysis.
Table 1.
Altmetric Attention Score Calculation
| Social Media Outlet | Weight |
|---|---|
| News | 8.0 |
| Blog | 5.0 |
| Policy document | 3.0 |
| Patent | 3.0 |
| Wikipedia | 3.0 |
| 1.0 | |
| Peer review (Publon, PubPeer) | 1.0 |
| Google+∗ | 1.0 |
| F1000 | 1.0 |
| Weibo† | 1.0 |
| Syllabi (Open Syllabus) | 1.0 |
| 0.5 | |
| 0.25 | |
| 0.25 | |
| YouTube | 0.25 |
| 0.25 | |
| Q&A (Stack Overflow) | 0.25 |
| Mendeley readers | 0 (tracked but not considered in calculation) |
Not trackable since 2019, but historical data kept. The total score is calculated as a function of the social media weights listed in the table.
Not trackable since 2015, but historical data kept.
Secondary bibliometric factors pertaining to each study were collected in accordance with previous studies. These article characteristics included journal, article title, date of publication, highest degree obtained by the first author, number of authors, number of academic institutions, geographic region of origin of the publication, open access status of the publication, disclosure of any conflict of interest (presence or absence of general self-reported conflict of interest), subject of study, study design, LOE, and the number of referenced studies. Similar to previous Altmetric investigations, study design was classified as either original clinical research (prospective/retrospective cohort, case–control, case series/report, randomized controlled trial [RCT]), systematic review or meta-analysis, review articles, basic science or biomechanics, or other. Article topics were basic science/biomechanics, diagnostics/imaging, epidemiology/etiology, treatment (including injections or surgical interventions), rehabilitation/return to play, outcomes, and other. The institutional affiliation of the corresponding author was used to identify the geographic origin of the article from either the United States, Europe, or other. The LOE was determined by the Center for Evidence-Based Medicine criteria, with RCTs and systematic review/meta-analysis of Level I studies constituting Level I, nonrandomized cohort studies constituting Level III studies, case series representing Level IV studies, and editorials and expert opinions constituting Level V.14
Statistical Analysis
All statistical analyses were performed with Stata, version 16.1 (StataCorp, College Station, TX). Normality was verified with the Shapiro–Wilks test and was not achieved for any of the included variables. Bibliometric and Altmetric characteristics were summarized by descriptive statistics using means with standard deviations for continuous variables and frequencies with percentages for categorical variables. A multivariable linear regression analysis was conducted to determine the association between article characteristics and AAS. Statistical significance was defined as P < .05.
Results
The Altmetric Database query yielded 619 studies published between 1990 and 2022. After screening for articles that fit our inclusion criteria, this investigation used AAS to identify and sequentially identify the top 100 most impactful articles generating online attention that pertained to the labrum of the shoulder. A total of 47 articles were excluded from the search, including 27 publications that were unrelated to the labrum and 20 that did not focus on labral pathologies. The median number of years since publication was 5 years (interquartile range [IQR] 1-9 years). The overall mean and standard deviation for the AAS was 24.85 ± 55.51, with a range of 7 to 460 (median: 11). The included articles represented 35 journals, with 62 articles attributed to 3 journals and their open access companions: American Journal of Sports Medicine (AJSM; 33%), Arthroscopy: The Journal of Arthroscopic & Related Surgery (Arthroscopy; 19%), and the Journal of Shoulder and Elbow Surgery (JSES; 10%). Most studies were Level IV evidence (32%), followed by Level III (16%). A large proportion of studies did not have a designated LOE (27%) and included literature reviews, infographics, and basic science/biomechanical investigations. Eleven articles were Level I evidence. The top 4 articles identified in this study had AAS scores that were substantially greater (AAS ≥160) than all other included articles, which were substantially lower (AAS ≤82). Three of the 4 articles with the greatest AAS included 2 meta-analyses and 1 RCT (Level I evidence), whereas the fourth was a diagnostic imaging study (Level IV). Two of the articles were published in the British Journal of Sports Medicine, one in AJSM, and the other in American Journal of Roentgenology. A complete summary of all included bibliometric and Altmetric characteristics can be found in Appendix Table 1 and Appendix Table 2, respectively, both available at www.arthroscopyjournal.org. The most common article type was original clinical research (73%), with the most common subgroup being a case series (26; 36% of original research articles). The most common article topics were diagnostics/imaging (33%) and treatment of labral pathologies (33%). Of the included studies, 64% originated from within the United States, 21% originated from Europe, and 15% were published outside of the United States and Europe. A large portion of the included articles specifically focused on SLAP tears (41%).
A total of 3193 Twitter mentions (median 13, IQR: 3-23) and 280 Facebook mentions (median 0, IQR 0-2) were identified from the 100 included articles. A multivariable linear regression model incorporating all bibliometric characteristics collected from included studies was constructed to determine the influence of each factor on the AAS (Table 2). This model demonstrated only LOE was statistically significant and associated with the AAS, with an additional mean increase in the AAS of 9.63 (95% confidence interval 2.29-16.95; P = .011) for every decrease in an article’s numerical LOE value. The regression analysis demonstrated that citation rate was not significantly associated with the AAS (P > .05).
Table 2.
Multivariate Linear Regression Model for Key Publication Characteristics, Citations, and the Altmetric Attention Score.
| Variable | Regression Coefficient | 95% Confidence Interval | P Value |
|---|---|---|---|
| Reference: subscription access | |||
| Open access | 22.83 | –4.29 to 49.95 | .098 |
| Reference: no general COI | |||
| COI | –11.24 | –36.35 to 13.86 | .376 |
| Reference: Non-M.D./D.O.∗ | |||
| M.D. or D.O. | 14.02 | –15.15 to 43.18 | .342 |
| Reference: not from North America | |||
| Continent of origin: North America | 9.36 | –16.66 to 35.36 | .477 |
| Reference: before 2017 | |||
| Year: after and including 2017 | 2.09 | –24.21 to 28.39 | .875 |
| Number of authors | –2.35 | –8.24 to 3.53 | .429 |
| Number of institutions | 9.12 | –1.53 to 19.78 | .092 |
| Level of Evidence | –7.74 | –15.36 to –0.12 | .047† |
| Journal impact factor | 0.65 | –0.08 to 1.38 | .080 |
| Number of references | 0.25 | –0.29 to 0.79 | .365 |
| Number of citations | 0.01 | –0.06 to 0.08 | .686 |
COI, conflict of interest.
Other author degrees include B.S., M.S., Ph.D., M.D. Ph.D., M.D. M.B.A., M.D. M.S., M.D. M.P.H., and P.T.
Indicates statistical significance at P < .05 level.
Discussion
Original clinical research studies performed in the United States or Europe published in 1 of 3 sports medicine journals receive the most online attention, as measured by AAS, of all studies on the shoulder labrum. The only variable associated with greater AAS was a decreasing numerical LOE.
The majority (57%) of most viewed labral papers identified here were published in 3 journals: AJSM, Arthroscopy, and JSES. The current 5-year IFs for these journals are 8.076,15 4.433,16 and 3.883,17 respectively. This represented the 2nd, 6th, and 24th highest IF out of the 86 orthopaedic journals, respectively. Preferential social media promotion of articles from well-known journals mirrors trends depicted previously in the literature for the ACL in which 65% of articles were published in AJSM, British Journal of Sports Medicine, or Journal of Orthopaedic & Sports Physical Therapy11 and MUCL in which 72% were AJSM, Orthopaedic Journal of Sports Medicine, or JSES.13 Conflicting orthopaedic literature exists on whether an association exists between journal IF and AAS, with some studies reporting no association,18,19 including ours, and others demonstrating an association.20,21
Of top labral papers, 64% were performed in the United States and 21% were from Europe. This mirrors what has been reported on the ACL and MUCL top articles by AAS, with a majority of articles by authors based in the United States.11,13 In contrast, more top rotator cuff articles were conducted in Europe (44%) than in the United States (25%).12 Further, when only looking at articles published in 5 English-language orthopaedic journals (3 North American, 2 European), Kunze et al.21 found that publication in North America was associated with greater AAS score, although the generalizability of this finding to our article collection, which includes articles from 35 different journals, is limited. In contrast, we report no association with country of origin and AAS score (P = .577).
Seventy-three percent of articles included were clinical research studies, which aligns with literature on ACL and MUCL, in which original research represented 60% and 74% of their cohort.11,13 When stratified by LOE, nearly half of studies were Level III or IV evidence (48%), whereas Level I and Level II evidence represent 11% and 6% of studies, respectively. This represents a lower numerical LOE than reported in the top MUCL papers, in which no level I studies were represented,13 but similar results to the top rotator cuff articles in which meta-analysis and randomized control trials represent 29% and 15% of studies, respectively.13 Multivariate regression model demonstrated a significant correlation between decreasing numerical LOE (i.e., greater LOE) and AAS. These findings suggest that greater-quality research studies on the shoulder labrum are being disseminated more on social media than studies with lower LOEs.
Citation rate was not associated with AAS. In contrast, a positive association between AAS and citations has been reported for general orthopaedic literature,20,22 arthroplasty,20 and hand.23 However, these analyses excluded studies within the last 2.5 to 4 years to allow for more representative data on citation counts, which may be delayed years past the date of publication. Of our top 100 papers identified, 11 were published in the year before analysis (2022) and, of these, only 3 had accrued citations in that time. The inclusion of these studies in our analysis may have prevented us from identifying an association between AAS score and citation count. Parrish et al.19 reported that, when including recent articles, only 2 articles were included on both the top 100 spine articles by AAS and top 100 spine articles by citation count over the same period. Here we opted to include all studies published at the time of analysis because it takes advantage of one of the most commonly cited advantaged of the AAS—the prompt availability of a metric for an article’s impact.24,25 The inclusion of these studies may provide a better representation of the current top shoulder labrum articles.
Limitations
There are several limitations to this study and to the use of AAS. First, though our study identified papers ranging from 1990 to 2022, there is an inherent bias toward increased online attention for more recent studies. A review of articles from 3 shoulder and elbow journals found that more than 50% of article tweets occurred between the time of online and print publication.26 Thus, the top articles identified here do not represent the most important studies on the shoulder labrum; rather, it is a representation of those studies that have obtained the most online attention. Second, AAS has been criticized as a reflection of the popularity of an article to the public, but not the quality of the research.27,28 In our study, we control for the popularity of the research topic through our inclusion criteria and look at other variables associated with articles, but we are unable to evaluate other factors that could reflect popularity as opposed to research quality, such as prestige of institution performing the research or novelty of presented results. Despite an article having a high AAS, readers must remain critical of the literature to determine the quality of their conclusions. Lastly, many orthopaedic journals and authors also use Instagram to disseminate their research to colleagues, but Instagram is not included in the AAS. These articles may be underrepresented here.
Conclusions
Top articles on the shoulder labrum, as defined by high AAS score, are most commonly original clinical research published in 1 of 3 sports medicine journals and performed in the United States or Europe. A decreased numerical LOE is associated with an increase in AAS score, but there is no association between AAS score and citation rate.
Disclosure
The authors (A.S.V., L.F., W.J., A.C., A.H., F.P.T., K.B.F.) report no conflicts of interest in the authorship and publication of this article. Full ICMJE author disclosure forms are available for this article online, as supplementary material.
Supplementary Data
References
- 1.Hirsch J.E. An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA. 2005;102:16569–16572. doi: 10.1073/pnas.0507655102. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.James C. Google Scholar Citations Open to All. Google. https://scholar.googleblog.com/2011/11/google-scholar-citations-open-to-all.html Published November 16, 2011.
- 3.Chapman C.A., Bicca-Marques J.C., Calvignac-Spencer S., et al. Games academics play and their consequences: H authorship, h-index and journal impact factors are shaping the future of academia. Proc R Soc B Biol Sci. 2019;286 doi: 10.1098/rspb.2019.2047. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hicks D., Wouters P., Waltman L., de Rijcke S., Rafols I. Bibliometrics: The Leiden Manifesto for research metrics. Nature. 2015;520:429–431. doi: 10.1038/520429a. [DOI] [PubMed] [Google Scholar]
- 5.Moorhead S.A., Hazlett D.E., Harrison L., Carroll J.K., Irwin A., Hoving C. A new dimension of health care: Systematic review of the uses, benefits, and limitations of social media for health communication. J Med Internet Res. 2013;15:e85. doi: 10.2196/jmir.1933. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kunze K.N., Vadhera A., Purbey R., Singh H., Kazarian G.S., Chahla J. Infographics are more effective at increasing social media attention in comparison with original research articles: An Altmetrics-based analysis. Arthroscopy. 2021;37:2591–2597. doi: 10.1016/j.arthro.2021.03.056. [DOI] [PubMed] [Google Scholar]
- 7.Elmore S.A. The Altmetric Attention Score: What does it mean and why should I care? Toxicol Pathol. 2018;46:252–255. doi: 10.1177/0192623318758294. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Kwok R. Research impact: Altmetrics make their mark. Nature. 2013;500:491–493. doi: 10.1038/nj7463-491a. [DOI] [PubMed] [Google Scholar]
- 9.Kunze K.N., Richardson M., Bernstein D.N., Premkumar A., Piuzzi N.S., McLawhorn A.S. Altmetrics Attention Scores for randomized controlled trials in total joint arthroplasty are reflective of high scientific quality: An Altmetrics-based methodological quality and bias analysis. JAAOS Glob Res Rev. 2020;4 doi: 10.5435/JAAOSGlobal-D-20-00187. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Djulbegovic M., Kalahasty K., Watane A., Jabori S.K., Al-Khersan H., Sridhar J. Correlation between Altmetric attention scores and citations for articles published in high-impact factor ophthalmology journals from 2018 to 2019. JAMA Ophthalmol. 2022;140:623. doi: 10.1001/jamaophthalmol.2022.0858. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Civilette M.D., Rate W.R., Haislup B.D., et al. The top 100 most impactful articles on the anterior cruciate ligament: An altmetric analysis of online media. SAGE Open Med. 2022;10 doi: 10.1177/20503121221111694. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Haislup B.D., Rate W.R., Civilette M.D., Cohen A.S., Bodendorfer B.M., Gould H.P. The 100 most impactful articles on the rotator cuff: An Altmetric analysis of online media. J Exp Orthop. 2022;9:92. doi: 10.1186/s40634-022-00530-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kirloskar K.M., Civilette M.D., Rate W.R., et al. The 50 most impactful articles on the medial ulnar collateral ligament: An altmetric analysis of online media. SAGE Open Med. 2022;10 doi: 10.1177/20503121221129921. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Howick J. The Oxford 2011 Levels of Evidence. The Centre for Evidence-Based Medicine. http://www.cebm.net/index.aspx?o=5653* Published 2011.
- 15.The American Journal of Sports Medicine. https://journals.sagepub.com/home/ajs
- 16.Arthroscopy Journal. https://www.arthroscopyjournal.org/
- 17.Journal of Shoulder and Elbow. https://www.jshoulderelbow.org/
- 18.Collins C.S., Singh N.P., Ananthasekar S., Boyd C.J., Brabston E., King T.W. The correlation between Altmetric score and traditional bibliometrics in orthopaedic literature. J Surg Res. 2021;268:705–711. doi: 10.1016/j.jss.2021.07.025. [DOI] [PubMed] [Google Scholar]
- 19.Parrish J.M., Jenkins N.W., Brundage T.S., Hrynewycz N.M., Singh K. The top 100 spine surgery articles on social media: An Altmetric study. Spine (Phila Pa 1976) 2020;45:1229–1238. doi: 10.1097/BRS.0000000000003508. [DOI] [PubMed] [Google Scholar]
- 20.Kunze K.N., Vadhera A.S., Polce E.M., et al. The Altmetric Attention Score is associated with citation rates and may reflect academic impact in the total joint arthroplasty literature. HSS J. 2023;19:37–43. doi: 10.1177/15563316221115723. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Kunze K.N., Polce E.M., Vadhera A., et al. What is the predictive ability and academic impact of the Altmetrics Score and social media attention? Am J Sports Med. 2020;48:1056–1062. doi: 10.1177/0363546520903703. [DOI] [PubMed] [Google Scholar]
- 22.Khalid M.A., Verma A., Kazan O., Chen J. Twitter use among orthopedic surgery journals correlates with increased citation rates. Orthopedics. 2023:46. doi: 10.3928/01477447-20221024-02. [DOI] [PubMed] [Google Scholar]
- 23.Yoshimura R., Grant M.C., Gardiner M.D., Wade R.G. Disseminating hand surgery research using social media: The relationship between Altmetrics and citations. J Hand Surg. 2021;46:740–747. doi: 10.1016/j.jhsa.2021.03.028. [DOI] [PubMed] [Google Scholar]
- 24.Baheti A.D., Bhargava P. Altmetrics: A measure of social attention toward scientific research. Curr Probl Diagn Radiol. 2017;46:391–392. doi: 10.1067/j.cpradiol.2017.06.005. [DOI] [PubMed] [Google Scholar]
- 25.Zhang D., Earp B.E. Correlation between social media posts and academic citations of orthopaedic research. JAAOS Glob Res Rev. 2020;4 doi: 10.5435/JAAOSGlobal-D-20-00151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Sudah S., Faccone R.D., Nasra M.H., Constantinescu D., Menendez M.E., Nicholson A. Twitter mentions influence academic citation count of shoulder and elbow surgery publications. Cureus. 2022;14 doi: 10.7759/cureus.21762. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Crotty D. Altmetrics. Eur Heart J. 2017;38:2647–2648. doi: 10.1093/eurheartj/ehx447. [DOI] [PubMed] [Google Scholar]
- 28.Sahu D., Poduval M. Altmetrics and impact factor: Relevance to the orthopaedic community. Indian J Orthop. 2022;56:1109–1111. doi: 10.1007/s43465-022-00659-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.