Abstract
Study Design:
Observational study.
Objectives:
To determine the publication rate of podium presentations from the North American Spine Society (NASS) annual meetings from the years 2009 to 2011.
Methods:
In April 2015, a PubMed search was conducted using titles from the paper presentations as well as the authors. Of the search results that were found, the specific scientific journal in which the article was published was recorded. We analyzed further the top 4 destination journals and trends in publications in these journals over the study period. No study funding was obtained for this research, and there are no potential conflicts of interest or associated biases.
Results:
Over the study period, 671 paper presentations were available and 342 were published (51% publication rate). The highest publication rate was from the 2011 annual meeting, with 55.3%, and the lowest year was 2010, with a rate of 46.43%. Spine (32.75%), The Spine Journal (19.01%), Journal of Neurosurgery Spine (7.31%), and European Spine Journal (6.73%) were the top 4 destination journals. Over the study period, we found a significant decrease in publication rate in Spine (P = .001) and a significant increase in publication rate in The Spine Journal (P = .003). There were no significant difference in publication rate over the study period in Journal of Neurosurgery Spine (P = .15) or European Spine Journal (P = .23).
Conclusions:
This is the first study to our knowledge evaluating the publication rate of podium presentations from recent North American Spine Society annual meetings. We found an overall publication rate of 51%.
Keywords: North American Spine Society, research, publication rate, annual meeting, podium presentation
Introduction
Annual society meetings serve as a platform for investigators and clinicians to discuss research efforts in their respective fields. Previous studies have noted publication rates of 34% to 66% following podium presentations for spine, orthopedic, and other subspecialty meetings.1–26 The North American Spine Society (NASS) was founded in 1984 to improve the quality of scientific knowledge and to promote excellence in patient care among professionals in the field of spinal disorders.27
NASS holds an annual meeting in the fall of each year and is an important forum for sharing research findings. To date one study has evaluated the publication from NASS annual meetings (1990 to 1992).1 Over the past 25 years, NASS has significantly grown in membership and the size of its annual meeting. Our objective was to evaluate the publication rate of podium presentations from the 2009 to 2011 NASS annual meetings.
Material and Methods
Paper presentations from the 2009 to 2011 annual NASS meetings were identified. The number of abstracts presented from 2009 to 2011 was recorded in a spreadsheet. PubMed, maintained by the United States National Library of Medicine at the National Institutes of Health was searched in April 2015. PubMed searches contained titles from the paper presentations, as well as the authors involved. All searches were limited to the English language.
For podium presentations that were published and identified in PubMed, we recorded the destination journal, the year of publication, and the length of time (in years) from presentation to publication. A 3-year follow-up period after the 2011 annual meeting was allowed as prior studies suggest that a majority of articles are published by this time.1,17,19,23 The publication rate of podium presentations per annual NASS meeting was calculated as well as the overall publication rate from the 3 annual meetings. The rate of publications per year for the top 4 publication journals were recorded. The impact factor of the publication journals was recorded for the year 2014 using the Thomson Reuters Journal Citations Report (Thomson Reuters, New York, NY).
Statistical Analysis
The t test was used to compare publication time from annual meeting to journal publication between the current study and a prior study evaluating the 1990 to 1992 NASS annual meetings. A one-way analysis of variance was used to test for differences in publication rates in journals over the study period. Significance was set at P < .05.
Results
Publication Rate
Over the 3-year study period, 671 podium presentations were available. The overall publication rate during the study period was 51% (342 of 671 presentations). The 2011 annual meeting had the highest publication rate (55.3%), and the 2010 annual meeting had the lowest publication rate 46.4% (Table 1). The majority of presentations (88.1%) were published within 3 years (Table 2), which was not significantly different (P = .13) as compared to a prior study evaluating publication rates of the 1990 to 1992 NASS annual meetings.1
Table 1.
Publication Rate for NASS Podium Presentations From 2009 to 2011.
| Meeting Year | Number of Podium Presentations | Published | Unpublished | Publication Rate |
|---|---|---|---|---|
| 2009 | 212 | 108 | 104 | 50.9% |
| 2010 | 224 | 104 | 120 | 46.4% |
| 2011 | 235 | 130 | 105 | 55.3% |
| Total | 671 | 342 | 329 | 51% |
Abbreviation: NASS, North American Spine Society.
Table 2.
Publication Time From Annual Meeting to Journal Publication.
| Time From Meeting (Years) | NASS Publications, 1990-19 92a, n (%) | NASS Publications, 2009-2011a, n (%) | P Value |
|---|---|---|---|
| 0 | 26 (11.7) | 47 (14.6) | .13 |
| 1 | 115 (64.1) | 113 (49) | |
| 2 | 45 (84.4) | 99 (78.8) | |
| 3 | 16 (91.8) | 47 (93) | |
| 4 | 12 (97.3) | 22 (99.6) | |
| >4 years | 6 (100) | 4 (100) |
Abbreviation: NASS, North American Spine Society.
aTen articles were published prior to annual meeting presentation.
Destination Journals and Impact Factor (Table 3)
Table 3.
Destination Journals for NASS Podium Presentations, 2009 to 2011.
| Journal | Number of Publications (n = 342) | % Total of Publications | Impact Factor (Year 2014) |
|---|---|---|---|
| Spine | 112 | 32.75% | 2.297 |
| The Spine Journal | 65 | 19.01% | 2.426 |
| Journal of Neurosurgery Spine | 25 | 7.31% | 2.383 |
| European Spine Journal | 23 | 6.73% | 2.066 |
| Journal of Bone and Joint Surgery | 19 | 5.56% | 5.280 |
| Journal of Spinal Disorders and Techniques | 13 | 3.80% | 2.202 |
| Neurosurgery | 10 | 2.92% | 3.620 |
| International Journal of Spine Surgery | 6 | 1.75% | NA |
| Journal of Trauma | 4 | 1.17% | 2.961 |
| Journal of Clinical Neuroscience | 4 | 1.17% | 1.378 |
| Orthopedics | 3 | 0.88% | 0.962 |
| Global Spine Journal | 3 | 0.88% | NA |
| American Journal of Orthopedics | 3 | 0.88% | NA |
| Clinical Neurosurgery | 2 | 0.58% | NA |
| Canadian Journal of Surgery | 2 | 0.58% | 1.507 |
| Lancet Oncology | 2 | 0.58% | 24.690 |
| World Neurosurgery | 2 | 0.58% | 2.878 |
| Journal of Pediatric Orthopedics | 2 | 0.58% | 1.474 |
| Journal of Orthopedic Research | 2 | 0.58% | 2.986 |
| PM&R | 2 | 0.58% | 1.534 |
| Evidence-based Spine-care Journal | 2 | 0.58% | NA |
| Lancet | 1 | 0.29% | 45.217 |
| Journal of Manual and Manipulative Therapy | 1 | 0.29% | NA |
| Advances and Technical Standards in Neurosurgery | 1 | 0.29% | NA |
| American Journal of Physical Medicine and Rehabilitation | 1 | 0.29% | 2.202 |
| Neurosurgical Focus | 1 | 0.29% | 2.105 |
| Korean Neurosurgery | 1 | 0.29% | NA |
| Injury | 1 | 0.29% | 2.137 |
| Pain Medicine | 1 | 0.29% | 2.339 |
| Zhonghua Yi Xue Za Zhi | 1 | 0.29% | NA |
| Osteoarthritis and Cartilage | 1 | 0.29% | 4.165 |
| Journal of Neurosurgery Pediatrics | 1 | 0.29% | 1.370 |
| Anesthesiology | 1 | 0.29% | 5.879 |
| Asian Spine Journal | 1 | 0.29% | NA |
| Artificial Organs | 1 | 0.29% | 2.050 |
| Risk Management and Healthcare Policy | 1 | 0.29% | NA |
| Journal of Bone and Mineral Research | 1 | 0.29% | 6.832 |
| International Wound Journal | 1 | 0.29% | 2.150 |
| Scoliosis | 1 | 0.29% | 1.590 |
| Clinics in Orthopedic Surgery | 1 | 0.29% | NA |
| Clinical Orthopedics and Related Research | 1 | 0.29% | 2.765 |
| Healthcare Policy | 1 | 0.29% | NA |
| Cancer | 1 | 0.29% | 5.068 |
| Neurological Sciences | 1 | 0.29% | 1.447 |
| Journal of Neurosurgical Science | 1 | 0.29% | 1.158 |
| The Bone and Joint Journal | 1 | 0.29% | 1.961 |
| Medical Physics | 1 | 0.29% | 2.635 |
| PLoS One | 1 | 0.29% | 3.234 |
| Journal of Orthopedic Surgery and Research | 1 | 0.29% | 1.386 |
| Journal of Physiotherapy | 1 | 0.29% | 3.708 |
| Geriatric Medical Science | 1 | 0.29% | NA |
| SAS Journal | 1 | 0.29% | NA |
| Pain | 1 | 0.29% | 5.213 |
| Obesity | 1 | 0.29% | 3.734 |
| Osteoporosis International | 1 | 0.29% | 4.169 |
| Asian Journal of Neurosurgery | 1 | 0.29% | NA |
| Pain Physician | 1 | 0.29% | 3.542 |
Abbreviations: NASS, North American Spine Society; NA, not available.
We identified 57 destination journals. Spine (32.75%), The Spine Journal (19.01%), Journal of Neurosurgery Spine (7.31%), and European Spine Journal (6.73%) were the top 4 destination journals. Ten journals accounted for 82.16% of the publications. The impact factor for each journal was queried, and 41 journals (72%) had impact factors available while 16 journals (28%) did not have impact factors. The median impact factor was 2.383, and the mean was 4.358 (0.962-45.217).
Changes in Publication Rate Among the Top 4 Destination Journals (Table 4)
Table 4.
Changes in Destination Journal Over the Study Period.
| Journal | 2009 Meeting Publications, n (%) | 2010 Meeting Publications, n (%) | 2011 Meeting Publications, n (%) | P Value |
|---|---|---|---|---|
| Spine | 50 (46.3%) | 29 (27.9%) | 33 (25.4%) | .001 |
| Spine Journal | 10 (9.6%) | 28 (26.9%) | 27 (20.8%) | .003 |
| Journal Neurosurgery Spine | 5 (4.6%) | 6 (5.8%) | 14 (10.8%) | .15 |
| European Spine Journal | 4 (3.7%) | 10 (9.6%) | 9 (6.9%) | .23 |
Over the study period, we found a significant decrease in publication rate in Spine (P = .001) and a significant increase in publication rate in The Spine Journal (P = .003). There were no significant differences in publication rate over the study period in Journal of Neurosurgery Spine (P = .15) or European Spine Journal (P = .23).
Discussion
The NASS was founded in 1984 to improve the quality of scientific knowledge and to promote excellence in patient care among professionals in the field of spinal disorders.27 Each year, the society holds an annual meeting where multidisciplinary spine specialists present and discuss research efforts.27
Publication of one’s work in a peer-reviewed journal following a presentation at a national meeting is ideal and important for the advancement of knowledge in an author’s field of study. In the neurosurgical literature, Patel et al demonstrated a higher publication rate following podium presentations (41%) as opposed to poster presentations (29%), suggesting the higher quality or greater impact of the podium presentations.19 In the pediatric orthopedic literature, Amirhamzeh et al also found differences in podium and poster presentation publication rates, with podium presentations having a 1.47 times greater likelihood of becoming published.4 Recently, Kinsella et al also found a higher publication rate of podium presentations as compared to poster presentations (73.3% vs 56.9%) at the American Orthopaedic Society for Sports Medicine meetings.28 We limited our study to podium presentations due to the higher likelihood of these being published as compared to posters, as noted by prior authors. This methodology is similar to a recent study by Okafor at al, which evaluated podium presentations only and found a 65.8% publication rate of podium presentations from the 2007 to 2011 Cervical Spine Research Society annual meetings.18
Sprague and colleagues identified 3 common barriers for manuscript publication following an abstract presentation at an international orthopedic meeting.29 These included a lack of time for manuscript preparation/completion, the fact that research was still ongoing for some studies, and the problematic relationships between coauthors.29 With longer follow-up, it is plausible more podium presentations from the 2009 to 2011 annual meetings will be submitted for publication and potentially published. However, our follow-up was more than 5 years for the 2009 study and 51% of these papers were published. It is unlikely a significantly greater difference in publication rate would arise with longer follow-up.
Wang et al performed an analysis of podium presentations for the 1990 to 1992 annual NASS meetings in 1999.1 The results demonstrated a 40.1% publication rate of the 545 presentations (poster and podium presentations) over that study period, which was comparable to 2 other major spine societies at the time.1 Even with the exclusion of poster presentations, our results reveal an increase of at least 23% (671 podium vs 545 total presentations) in the volume of presentations since the Wang et al study, and an increase in the publication rate by nearly 10.9% (51% vs 40.1%). Albeit our study’s publication rate would most likely be lower if poster publication rates were included.
Our results continue to demonstrate that most articles (91.8% vs 88.1%) are published within 4 years of presentation (Table 2). Additionally, there was no statistical difference (P = .13) as compared to the prior study1 on publication time from annual meeting to journal publication. Our comparative data with Wang et al should be interpreted with caution, as we did not take poster presentations into account. In addition, there were 10 studies that were published prior to presentation at the NASS meeting during the study period. However, other orthopedic specialty society meetings have produced similar results and publication rates.2,4,8,16,18,28
We also identified 57 destination journals for the podium presentations. Spine (32.75%), The Spine Journal (19.01%), Journal of Neurosurgery Spine (7.31%), and European Spine Journal (6.73%) were the top 4 destination journals. A recent study of publication rates from the Cervical Spine Research Society annual meeting also found Spine to be the most common destination journal (40.76%), followed by The Spine Journal (11.8%) and Journal of Neurosurgery Spine (11.8%).18 We found a significant decrease in publication rate in Spine (P = .001) and a significant increase (P = .003) in publication rate in The Spine Journal over the study period (Table 4). This may reflect the increase in the impact factor of The Spine Journal as well as NASS annual meeting podium presenters’ preferences to submit papers to the society’s journal. In the analysis of the 1990 to 1992 NASS annual meetings, 68.5% of publications were in Spine, 8.6% in Journal of Spinal Disorders and Techniques, and 0% in European Spine Journal. The Spine Journal did not start publication until 2001, and Journal of Neurosurgery Spine did not start publication until 2002 (it was a supplement of Journal of Neurosurgery prior to 2002), thus reflecting the lack publication in these journals. The increase in publication in European Spine Journal, from 0% to 6.73%, may reflect the annual meeting’s development of an international audience and international presenters. Moreover, the European Spine Journal did not start publication until 1992, and its initial readership was most likely geographically different than NASS annual meeting attendants.
The 2014 impact factor of the 57 destination journals was queried. We found 41 of the 57 journals to have identifiable impact factors. The average impact factor was 4.358, and the median was 2.383. In a study evaluating impact factors of publications from abstract at the annual meeting of the Spine Society of Europe (EuroSpine), the average impact factor was 1.798.2 While the impact factor of destination journals for NASS annual meeting presentations was higher compared to EuroSpine, it is concerning that 16 journals did not have identifiable impact factors. There have been concerns regarding the impact factor system with regard to how clinical research may be published in low impact factor journals that are not as highly cited as basic science journals. However, the clinical research is more impactful to the clinician rather than the basic science journals.30
In this study, we are focusing solely on a bibliometric analysis of podium presentations. However, future studies can evaluate the Altmetrics or “Alternative Metrics” of podium presentations and subsequent publications. Altmetrics incorporates the digital footprint of articles from social media sources, blogs, and Wikipedia, to name a few.31,32 Through the altmetric score that is generated the interest a podium presentation or publication generates can be further quantified. To date and to our knowledge no studies have evaluated altmetric scores of spine publications or abstracts presented at spine meetings such as NASS.
There are several limitations to our study. We relied on PubMed for our search of publications. However, papers published in non-PubMed cited journals would not have been identified and can affect the publication rate. There are also international authors and papers, and there is a possibility that due to our English language restrictions in the search criteria, we may have missed articles that were published in other languages. We conducted our research in April 2015 and with longer follow-up it is possible more presentations may be published. It is also possibly award winning papers or “Best Paper Designations” that started in the 2011 annual meeting may have had a higher publication rate as compared to non–best paper designated presentations. We did not evaluate this since we would have only a one annual meeting data point rather than the 3 annual meetings we evaluated.
Conclusions
In this first study to our knowledge evaluating the publication rate of podium presentations from recent NASS annual meetings, we found an overall publication rate of 51%. The 2011 annual meeting had the highest publication rate (55.3%), and the 2010 annual meeting had the lowest publication rate (46.4%). Spine (32.75%), The Spine Journal (19.01%), Journal of Neurosurgery Spine (7.31%), and European Spine Journal (6.73%) were the top 4 destination journals.
The ultimate goal of a podium presentation is dissemination of the research via peer-reviewed publication. The 51% rate of publications from the NASS annual meeting demonstrates the high quality of papers presented at this meeting.
Footnotes
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
References
- 1. Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major Spine Specialty Society meetings (NASS, SRS, ISSLS). Spine (Phila Pa 1976). 1999;24:425–427. [DOI] [PubMed] [Google Scholar]
- 2. Schulte TL, Huck K, Osada N, et al. Publication rate of abstracts presented at the Annual Congress of the Spine Society of Europe (years 2000-2003). Eur Spine J. 2012;21:2105–2112. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Varghese RA, Chang J, Miyanji F, Reilly CW, Mulpuri K. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America: is there a difference between accepted versus rejected abstracts? J Pediatr Orthop. 2011;31:334–340. [DOI] [PubMed] [Google Scholar]
- 4. Amirhamzeh D, Moor MA, Baldwin K, Hosalkar HS. Publication rates of abstracts presented at Pediatric Orthopaedic Society of North America meetings between 2002 and 2006. J Pediatr Orthop. 2012;32:e6–e10. [DOI] [PubMed] [Google Scholar]
- 5. Donegan DJ, Kim TW, Lee GC. Publication rates of presentations at an annual meeting of the American Academy of Orthopaedic Surgeons. Clin Orthop Relat Res. 2010;468:1428–1435. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Daluiski A, Kuhns CA, Jackson KR, Lieberman JR. Publication rate of abstracts presented at the annual meeting of the Orthopaedic Research Society. J Orthop Res. 1998;16:645–649. [DOI] [PubMed] [Google Scholar]
- 7. Kleine-Konig MT, Schulte TL, Gosheger G, Rodl R, Schiedel FM. Publication rate of abstracts presented at European Paediatric Orthopaedic Society annual meetings, 2006 to 2008. J Pediatr Orthop. 2014;34:e33–e38. [DOI] [PubMed] [Google Scholar]
- 8. DeMola PM, Hill DL, Rogers K, Abboud JA. Publication rate of abstracts presented at the shoulder and elbow session of the American Academy of Orthopaedic Surgery. Clin Orthop Relat Res. 2009;467:1629–1633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Nguyen V, Tornetta P, 3rd, Bkaric M. Publication rates for the scientific sessions of the OTA. Orthopaedic Trauma Association. J Orthop Trauma. 1998;12:457–459. [DOI] [PubMed] [Google Scholar]
- 10. Schoenfeld AJ, Carey PA, Frye BD, Weaver CR, Belmont PJ. Publication rate of presentations at an annual military orthopaedic meeting. J Surg Orthop Adv. 2012;21:204–209. [DOI] [PubMed] [Google Scholar]
- 11. Abicht BP, Donnenwerth MP, Borkosky SL, Plovanich EJ, Roukis TS. Publication rates of poster presentations at the American College of Foot and Ankle Surgeons annual scientific conference between 1999 and 2008. J Foot Ankle Surg. 2012;51:45–49. [DOI] [PubMed] [Google Scholar]
- 12. Whitehouse MR, Atwal NS, Blom AW. Publication rates for hip surgery-related abstracts presented at national and international meetings. Orthopedics. 2009;32:407. [DOI] [PubMed] [Google Scholar]
- 13. Jasko JJ, Wood JH, Schwartz HS. Publication rates of abstracts presented at annual Musculoskeletal Tumor Society meetings. Clin Orthop Relat Res. 2003;(415):98–103. [DOI] [PubMed] [Google Scholar]
- 14. Gavazza JB, Foulkes GD, Meals RA. Publication pattern of papers presented at the American Society for Surgery of the Hand annual meeting. J Hand Surg Am. 1996;21:742–745. [DOI] [PubMed] [Google Scholar]
- 15. Hoag CC, Elterman DS, Macneily AE. Abstracts presented at the American Urological Association annual meeting: determinants of subsequent peer reviewed publication. J Urol. 2006;176(6 pt 1):2624–2629. [DOI] [PubMed] [Google Scholar]
- 16. Abzug JM, Osterman M, Rivlin M, Paryavi E, Osterman AL. Current rates of publication for podium and poster presentations at the American Society for Surgery of the Hand annual meetings. Arch Bone Jt Surg. 2014;2:199–202. [PMC free article] [PubMed] [Google Scholar]
- 17. Bhandari M, Devereaux PJ, Guyatt GH, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002;84:615–621. [DOI] [PubMed] [Google Scholar]
- 18. Okafor L, Frost C, Mesfin A. Publication rate of paper presentations from the Cervical Spine Research Society annual meeting. Spine (Phila Pa 1976). 2015;40:699–702. [DOI] [PubMed] [Google Scholar]
- 19. Patel AJ, Cherian J, Fox BD, et al. Publication patterns of oral and poster presentations at the annual meetings of the Congress of Neurological Surgeons and the American Association of Neurological Surgeons. J Neurosurg. 2011;115:1258–1261. [DOI] [PubMed] [Google Scholar]
- 20. Ohtori S, Kubota G, Inage K, et al. English publication rate of 3,205 abstracts presented at the annual meeting of the Japanese Orthopaedic Association and the annual research meeting of the Japanese Orthopaedic Association. J Orthop Sci. 2013;18:1031–1036. [DOI] [PubMed] [Google Scholar]
- 21. Preston CF, Bhandari M, Fulkerson E, Ginat D, Koval KJ, Egol KA. Podium versus poster publication rates at the orthopaedic trauma association. Clin Orthop Relat Res. 2005;(437):260–264. [DOI] [PubMed] [Google Scholar]
- 22. Preston CF, Bhandari M, Fulkerson E, Ginat D, Egol KA, Koval KJ. The consistency between scientific papers presented at the Orthopaedic Trauma Association and their subsequent full-text publication. J Orthop Trauma. 2006;20:129–133. [DOI] [PubMed] [Google Scholar]
- 23. Scherer RW, Dickersin K, Langenberg P. Full publication of results initially presented in abstracts. A meta-analysis. JAMA. 1994;272:158–162. [PubMed] [Google Scholar]
- 24. Schnatz PF, Romegialli A, Abrantes J, et al. The North American Menopause Society: from abstract to publication. Menopause. 2008;15:996–1001. [DOI] [PubMed] [Google Scholar]
- 25. ul Haq MI, Gill I. Observational analysis of BOA free-papers (2001): from presentation to publication and comparison with the American Academy of Orthopaedic Surgeons (AAOS). Injury. 2011;42:418–420. [DOI] [PubMed] [Google Scholar]
- 26. Yoon PD, Chalasani V, Woo HH. Conversion rates of abstracts presented at the Urological Society of Australia and New Zealand (USANZ) annual scientific meeting into full-text journal articles. BJU Int. 2012;110:485–489. [DOI] [PubMed] [Google Scholar]
- 27. North American Spine Society bylaws. https://www.spine.org/Documents/WhoWeAre/Bylaws.pdf. Accessed April 21, 2015.
- 28. Kinsella SD, Menge TJ, Anderson AF, Spindler KP. Publication rates of podium versus poster presentations at the American Orthopaedic Society for Sports Medicine meetings: 2006-2010. Am J Sports Med. 2015;43:1255–1259. [DOI] [PubMed] [Google Scholar]
- 29. Sprague S, Bhandari M, Devereaux PJ, et al. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am. 2003;85:158–163. [DOI] [PubMed] [Google Scholar]
- 30. Kodumuri P, Ollivere B, Holley J, Moran CG. The impact factor of a journal is a poor measure of the clinical relevance of its papers. Bone Joint J. 2014;96:414–419. [DOI] [PubMed] [Google Scholar]
- 31. Warren HR, Raison N, Dasgupta P. The rise of altmetrics. JAMA. 2017;317:131–132. [DOI] [PubMed] [Google Scholar]
- 32. Butler JS, Kaye ID, Sebastian AS, et al. The evolution of current research impact metrics: from bibliometrics to altmetrics? Clin Spine Surg. 2017;30:226–228. doi:10.1097/BSD.0000000000000531. [DOI] [PubMed] [Google Scholar]