Abstract
Background
Open access (OA) articles are freely accessible online, either on the publisher/journal website or in a repository, a publicly available, free-of-charge online database. The primary aim of this study was to investigate whether OA publication confers a citation advantage in pediatric orthopaedics.
Methods
Pediatric orthopaedic studies published in English from January 1, 2012 to June 30, 2012 were identified through Excerpta medica database , Cochrane, and PubMed. Abstract screening and full-text evaluation were performed in duplicate. Citation counts over 10 years following publication, 2012 journal impact factor, OA status, type of OA, journal field, geographic location of senior author and journal publication, study design, study focus, subspecialty, level of evidence, and presence of funding were recorded. Statistical analyses were performed using independent samples t-tests, 1-way analysis of variance, χ2 tests, and multiple regression analysis.
Results
Of this study's 989 pediatric orthopaedic articles, 43.8% were OA. The mean citation count was 19.8 ± 24.4 on Web of Science. Compared to OA publications, the highest percentage of non-OA articles were published in a journal from North America, had a senior author from North America, were indexed in Journal Citation Reports, and were published in orthopaedic journals (P < .001). In multiple regression analysis, OA publication, higher levels of evidence, publication in a journal with a higher impact factor, having a senior author from Europe or North America, and study funding were associated with significantly increased citation counts. OA articles were cited an additional 3 times, on average, over 10 years.
Conclusions
Open access publication in pediatric orthopaedics confers an advantage of 3 extra citations over a decade, on average.
Key Concepts
-
(1)
Publishing pediatric orthopaedic articles open access confers an advantage of 3 additional citations over a decade compared to non-open access publication.
-
(2)
43.8% of pediatric orthopaedic articles are published open access.
Level of Evidence
III
Keywords: Open access, Citation rate, Publication trends
Introduction
Open access (OA) is defined as access to online journal articles without price or permission barriers [1]. Since the release of the Budapest Open Access Initiative in 2002, there has been an increasing trend in OA publishing [2]. While in 2006 there was a reported OA publication rate of 26% in biomedical literature, a more recent analysis of 2015 data suggests OA publication rates ranging from 30% to 70% depending on geographic location [3], [4]. With this shift in the publication paradigm, there has been ongoing debate regarding the effect of publishing OA articles on scientific impact, citation counts, and quality of evidence published. Some believe that many OA journals do not have proper peer-review systems in place, and that some “pay to publish” OA publishers may be more interested in monetary benefits rather than the quality of the scientific work. This may result in distrust from the scientific community [5]. Open access journals may have been further tarnished by the surge in predatory journals, which are journals from questionable publishers that exploit the OA model to extract fees from authors [6], [7], [8]. Nevertheless, it should be noted that many OA journals have been established as prestigious and reputable sources, such as the Public Library of Science journals. OA publishing may also positively impact access to research outcomes and clinical knowledge, especially in low-income countries [9], [10].
Freely available papers may be more likely to be accessed and cited. The European arm of the Scholarly Publishing and Academic Resources Coalition, an international alliance of academic libraries, compiled a list of studies published up until 2015 that investigated potential open access citation advantage (OACA) [11]. Of the 70 included studies, 65.7% found a citation advantage with OA publishing, and 34.3% were inconclusive or found no citation advantage. In a cross-sectional study comparing citation rates between OA and non-OA articles published in 2004 in Proceedings of the National Academy of Sciences, OA articles were twice as likely to be cited as non-OA articles [12]. In 2 randomized controlled trials that randomly assigned papers to OA or non-OA status, there were no reported differences in citation counts at 1 and 3 years after publication [13], [14]. A systematic review in 2021 found mixed results amongst 134 studies on OACA in various subjects, concluding that 71.6% of studies either confirmed the existence of OACA or found OACA in a subset of their sample [15]. The question of OACA appears to vary between disciplines, journals, and different periods of time and continues to be highly controversial in the medical research community [12], [13], [16], [17], [18], [19], [20], [21], [22], [23].
In 2014, 63 orthopaedic journals listed in Journal Citation Reports (JCR) were assessed; 20 (31.7%) offered authors the choice to publish OA for an author processing charge (APC), and 5 (7.9%) published all their articles OA [9]. The authors found no difference in the 2012 impact factor and volume of level 1 evidence between the 25 orthopaedic journals that published OA articles and the 38 purely subscription-based journals [9]. They concluded that there was equivalent importance and quality of articles published in OA and subscription-based orthopaedic journals. The presence of OA articles has greatly increased in the last 2 decades, with the emergence of OA journals as a major contributing force [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12]. A recent study on orthopaedic publications found a citation advantage for publishing in conventional journals compared to their associated OA journals. For example, higher citation counts were noted amongst articles published in American Journal of Sports Medicine compared to Orthopaedic Journal of Sports Medicine. However, the study was limited to the 5 highest-impact orthopaedic journals and their OA counterparts, resulting in possible selection bias by authors and journals with regard to the articles chosen to be published in the companion OA journals [24].
The primary objective of this study was to determine if a citation advantage exists for pediatric orthopaedic studies published OA compared to non-OA. It was hypothesized that OA studies would be more frequently cited than non-OA articles. Secondary objectives were to determine the prevalence of OA articles in pediatric orthopaedic and to evaluate the quality of OA versus non-OA studies based on the level of evidence.
Materials and methods
Search strategy and eligibility criteria
PubMed, Exerpta medica database (EMBASE), and Cochrane were systematically searched for published studies pertaining to pediatric orthopaedics published between the period of January 1, 2012 to June 30, 2012, inclusive. The year 2012 was selected to allow sufficient time to have passed for these articles to be cited (1 decade). A combination of keywords and medical subject headings (MeSH) terms were searched (Appendix, Fig. 1).
Fig. 1.
Preferred reporting items for systematic reviews and meta-analyses (PRISMA) diagram showing flow of study inclusion. PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Each manuscript underwent 2 phases of screening using the systematic review software Covidence (Veritas Health Innovation). Titles and abstracts were initially screened in duplicate, with a third reviewer resolving conflicts (JMS, CP, CW, SS, GW). Two reviewers (SS, GW) then independently assessed each full text for eligibility. Conflicts were resolved by the senior author (ED). Studies relevant to pediatric orthopaedics, in which the mean age of participants was ≤18 years, were included. Non-English studies, letters to the editor, editorials, commentary articles, abstracts, book chapters, and animal and basic science studies were excluded.
Data extraction
Two reviewers (GW, SS) extracted data from full texts in duplicate. For each included article, the following information was collected regarding the journal: journal name, whether it is JCR indexed, journal subspecialty (general orthopaedic, pediatric orthopaedic, other orthopaedic subspecialty etc.), and geographic location of publication. Each study was also evaluated for study design (randomized controlled trial, prospective cohort study, etc.), study focus (therapeutic, diagnostic, etc.), level of evidence, study subspecialty (trauma, spine, etc.), funding (yes, no), continent where the senior author works (geographic location of author), and number of centers involved (single center, multicenter). The level of evidence was evaluated according to the Oxford Center for Evidence-Based Medicine [25].
Each included study was searched for within Google Scholar, Scopus, and Web of Science (WoS), in a 1-week span within August 2022. Citation counts were identified in each database. Citation counts can vary between these sources due to differing metrics. For instance, WoS does not count citations from journals that are not indexed in their repository, books, conference proceedings, dissertations, patents, or technical reports. Given its conservative citation count, Web of Science was used in statistical analyses of citation counts for this study. If a manuscript was not listed in Web of Science, then it was not included in statistical analyses on citation counts.
Journal impact factor from 2012 was retrieved from JCR for 2012. For a given journal, JCR calculates the impact factor by dividing the total number of citations from articles in that journal published in the last 2 years by the total number of articles published in that journal over the last 2 years. For JCR-indexed journals, if the impact factor from 2012 was not available, the impact factor for the closest year was recorded. For journals not indexed in JCR, an impact factor of zero was assigned for the purpose of statistical analysis.
Assessment of OA status
A number of different definitions of OA have been proposed. Some are based on the presence of an OA creative common license, while others simply define OA as freely accessible online. For the purposes of this study, the definition by Piwowar and colleagues is used: OA publications are free to read online, either on the publisher/journal website or in a repository such as PubMed Central (PMC) [16]. Freely accessible “illegally” posted publications and preprint articles posted on other sources such as research or social media networks and author's personal websites were not considered OA.
OA versus non-OA status of each article was determined by independently searching each article on Google, Google Scholar, PubMed, and/or the journal/publisher site in April 2020 using a residential internet protocol address with no access to any library or electronic subscription services.
OA literature was classified into 1 of 4 categories as previously defined by Piwowar et al. in 2018 [16].
-
•
Gold: Published in an OA journal.
-
•
Hybrid: Available under an open license in a toll-access subscription journal. Authors may pay an APC in exchange for OA status in these subscription journals. Some articles are also elected for OA publishing at the discretion of the journal without a charge to the author.
-
•
Bronze: Similar to Hybrid, Bronze articles are free to read on the journal/publisher website but do not have a clearly identifiable OA license. This lack of identifiable license may not be intentional. However, without a license, the article legally does not allow extended reuse rights beyond reading.
-
•
Green: Accessible in an OA repository such as PMC, but toll-access on the publisher or journal page. Green articles may be self-archived or deposited in repositories by the publisher. Many funding agencies, including the National Institutes of Health, Howard Hughes Medical Institute, Wellcome Trust, and Research Councils UK, require that funded studies be deposited into a repository. Of note, publisher-hosted content (Gold, Hybrid, Bronze) takes priority over archived articles that would otherwise be classified as Green for purposes of classification.
OA classifications were determined by searching the publisher’s guidelines regarding OA status and by identifying the presence or absence of a copyright license for the article [26]. For example, if an article was available on the publisher's website, but carried no license information, it would be classified as Bronze rather than Hybrid. If an article was available in a repository but also available on a toll-access journal website, it would be classified as Bronze or Hybrid (depending on the availability of license information) rather than Green.
Statistical analysis
All data were analyzed using SPSS version 22.0 (IBM Corp). Summary statistics of all citation characteristics are reported as means and standard deviations for continuous characteristics and frequencies and percentages for discrete characteristics. A comparison of continuous characteristics between OA versus non-OA articles was performed using independent samples t-tests. Chi-square analysis was used to determine differences in categorical characteristics. One-way analysis of variance with Tukey post-hoc analysis was used to determine if mean citation count and impact factor differed between Gold, Hybrid, Bronze, and Green OA articles.
Multiple regression analysis was performed to identify the study characteristics associated with mean number of WoS citations between OA and non-OA studies while also controlling for other factors that might have influenced the outcome. Candidate variables for the regression analysis were selected a priori and included OA versus non-OA, level of evidence, journal impact factor, multicenter versus single center, whether or not a study was funded, JCR index status, and geographic location of senior author and journal. All analyses were performed using 2-sided hypothesis testing with statistical significance set to P ≤ .05.
Results
Characteristics of included articles
Seven thousand two hundred seventeen titles/abstracts were screened, of which 2449 full texts were further evaluated, and 989 articles were included in the final analysis (Fig. 1). Article and journal characteristics are presented in Table 1. Eight hundred sixty-two (87.2%) articles were original/primary research, and 127 (12.8%) were systematic/narrative reviews or meta-analyses. The majority of studies were case series by design (562, 56.8%) and were therapeutic in nature (516, 52.2%). Only 29(2.9%) articles were level 1 evidence. Study funding was reported in 176 (17.8%) articles. More than half of the studies were published in North America (514, 52.0%).
Table 1.
Characteristics of OA and non-OA articles included in the study.
| All articles (n = 989) |
OA articles (n = 433) |
Non-OA articles (n = 556) |
P value | |
|---|---|---|---|---|
| Citation count | ||||
| Google Scholar | 35.0 ± 53.0 (n = 980) |
35.5 ± 47.8 (n = 429) |
34.5 ± 56.8 (n = 551) |
.771 |
| Scopus | 20.7 ± 25.6 (n = 909) |
21.1 ± 28.9 (n = 384) |
20.4 ± 23.0 (n = 525) |
.700 |
| Web of Science | 19.8 ± 24.4 (n = 855) |
21.4 ± 28.7 (n = 326) |
18.7 ± 21.4 (n = 529) |
.144 |
| Article characteristics | ||||
| Study design | ||||
| Case series/case reports | 562 (57%) | 259 (60%) | 303 (54%) | .052 |
| Cross-sectional study | 140 (14%) | 48 (11%) | 92 (17%) | |
| Narrative review | 98 (10%) | 50 (12%) | 48 (9%) | |
| Case-control studies | 57 (6%) | 18 (4%) | 39 (7%) | |
| Retrospective cohort study | 52 5%) | 20 (5%) | 32 (6%) | |
| Prospective cohort study | 29 (3%) | 11 (3%) | 18 (3%) | |
| Systematic review | 26 (3%) | 15 (3%) | 11 (2%) | |
| Randomized controlled trial | 22 (2%) | 11 (3%) | 11 (2%) | |
| Meta-analysis | 3 (0%) | 1 (0%) | 2 (0%) | |
| Study focus | ||||
| Therapeutic | 516 (52%) | 230 (53%) | 286 (51%) | .820 |
| Diagnosis | 262 (26%) | 115 (27%) | 147 (26%) | |
| Prognosis | 154 (16%) | 65 (15%) | 89 (16%) | |
| Epidemiology | 45 (5%) | 18 (4%) | 27 (5%) | |
| Screening | 7 (1%) | 4 (1%) | 3 (1%) | |
| Economic and decision analysis | 5 (1%) | 1 (0%) | 4 (1%) | |
| Level of clinical evidence* | ||||
| 1 | 29 (3%) | 13 (3%) | 16 (3%) | .210 |
| 2 | 39 (4%) | 15 (3%) | 24 (4%) | |
| 3 | 256 (26%) | 99 (23%) | 157 (28%) | |
| 4 | 665 (67%) | 306 (71%) | 359 (65%) | |
| Senior author geographic location* | ||||
| North America | 357 (36%) | 122 (28%) | 235 (42%) | <.001 |
| Europe | 310 (31%) | 148 (34%) | 162 (29%) | |
| Asia | 254 (26%) | 133 (31%) | 121 (22%) | |
| South America | 18 (2%) | 11 (3%) | 7 (1%) | |
| Australia/New Zealand | 31 (3%) | 11 (3%) | 20 (4%) | |
| Africa | 19 (2%) | 8 (2%) | 11 (2%) | |
| Article subspecialty | ||||
| Trauma | 170 (17%) | 76 (18%) | 94 (17%) | .229 |
| Infection/Tumor | 148 (15%) | 80 (18%) | 68 (12%) | |
| Spine | 183 (19%) | 80 (18%) | 103 (19%) | |
| Sports | 69 (7%) | 24 (6%) | 45 (8%) | |
| Foot/Ankle | 74 (7%) | 25 (6%) | 49 (9%) | |
| Hand | 26 (3%) | 12 (3%) | 14 (3%) | |
| Hip | 70 (7%) | 27 (6%) | 43 (8%) | |
| Lower extremity | 64 (6%) | 29 (7%) | 35 (6%) | |
| Neuromuscular | 78 (8%) | 32 (7%) | 46 (8%) | |
| Upper extremity | 28 (3%) | 13 (3%) | 15 (3%) | |
| Other | 79 (8%) | 35 (8%) | 44 (8%) | |
| Centers involved* | ||||
| Single center | 872 (88%) | 383 (88%) | 489 (88%) | .808 |
| Multicenter | 117 (12%) | 50 (12%) | 67 (12%) | |
| Study funding* | ||||
| No funding | 813 (82%) | 349 (81%) | 464 (83%) | .245 |
| Funded | 176 (18%) | 84 (19%) | 92 (17%) | |
| Journal characteristics | ||||
| Published in journal indexed on journal of citation reports* | ||||
| Not indexed | 149 (15%) | 115 (27%) | 34 (6%) | <.001 |
| Indexed | 840 (85%) | 318 (73%) | 522 (94%) | |
| Impact factor* | ||||
| Impact factor (mean ± SD) | 1.7 ± 2.4 | 1.6 ± 2.9 | 1.7 ± 1.9 | .595 |
| Journal geographic location* | ||||
| North America | 514 (52%) | 134 (31%) | 380 (68%) | <.001 |
| Europe | 373 (38%) | 221 (51%) | 152 (27%) | |
| Asia | 77 (8%) | 60 (14%) | 17 (3%) | |
| South America | 7 (1%) | 7 (2%) | 0 (0%) | |
| Australia/New Zealand | 9 (1%) | 5 (1%) | 4 (1%) | |
| Africa | 9 (1%) | 6 (1%) | 3 (1%) | |
| Journal specialty | ||||
| Other orthopaedic subspecialty | 233 (24%) | 71 (16%) | 162 (29%) | <.001 |
| General orthopaedic | 231 (23%) | 148 (34%) | 83 (15%) | |
| Miscellaneous | 185 (19%) | 113 (26%) | 72 (13%) | |
| Pediatric orthopaedic | 148 (15%) | 34 (8%) | 114 (21%) | |
| Pediatrics | 61 (6%) | 15 (3%) | 46 (8%) | |
| Radiology | 54 (5%) | 25 (6%) | 29 (5%) | |
| Other surgical subspecialties | 51 (5%) | 10 (2%) | 41 (7%) | |
| Rehabilitation/Physical therapy | 26 (3%) | 17 (4%) | 9 (2%) | |
OA, open access; SD, standard deviation.
Included for multiple regression analysis.
Most articles were published in orthopaedic journals, including general, pediatric orthopaedic and other orthopaedic sub-specialty journals (612, 61.9% [Table 1]). 148 (15.0%) articles were published in pediatric orthopaedic journals, namely the Journal of Children’s Orthopaedics (JCO), Journal of Pediatric Orthopaedics (JPO), and Journal of Pediatric Orthopaedics B (JPO B).
Of the 989 manuscripts, 980 were listed in Google Scholar (citation count 35.0 ± 53.0), 909 were listed in Scopus (citation count 20.7 ± 25.6), and 855 were listed in WoS (citation count 19.8 ± 24.4). Eight hundred forty articles (84.9%) were published in journals indexed in JCR. The mean journal impact factor, assigning nonindexed journals an impact factor of zero, was 1.7 ± 2.4. The mean impact factor for only journals indexed in JCR was 1.9 ± 2.5.
Prevalence of OA publishing in pediatric orthopaedics
Of the 989 pediatric orthopaedic articles included in this study, 43.8% were OA. Of the 464 articles published in general and other orthopaedic subspecialty journals, 219 (47.2%) were OA. However, only 34 (23.0%) of the 148 articles published in the pediatric orthopaedic journals JCO, JPO, and JPO B were OA.
Of a total of 433 OA articles, the majority were gold OA (197, 45%), followed by Bronze (136, 31%), Green (92, 21%), and Hybrid (8, 2%). Green OA articles were most likely to be cited, while gold OA articles had the lowest citation counts (P < .001) (Table 2). Gold OA articles had a significantly lower mean citation count identified by Google Scholar, Scopus, and WoS compared to Bronze and Green OA articles (Table 3). Gold OA articles had a significantly lower mean journal impact factor compared to Bronze and Green OA. Green OA articles were published in journals with a lower impact factor, on average, than Bronze. No other significant differences were identified (Table 3).
Table 2.
Mean citation count by open access type.
| Type of open access |
P value | ||||
|---|---|---|---|---|---|
| Gold (n = 197) |
Hybrid (n = 8) |
Bronze (n = 136) |
Green (n = 92) |
||
| Mean number of citations | |||||
| Google Scholar | 23.8 ± 44.7 (n = 193) |
38.5 ± 30.9 (n = 8) |
44.6 ± 48.3 (n = 136) |
46.5 ± 49.7 (n = 92) |
<.001 |
| Scopus | 12.9 ± 23.6 (n = 175) |
25.4 ± 21.4 (n = 8) |
27.9 ± 30.9 (n = 114) |
28.2 ± 32.2 (n = 87) |
<.001 |
| Web of Science | 13.1 ± 24.3 (n = 123) |
18.6 ± 18.6 (n = 7) |
26.5 ± 29.3 (n = 118) |
27.3 ± 32.0 (n = 78) |
<.001 |
| Mean impact factor of journals (2012) | 0.8 ± 1.5 (n = 197) |
1.6 ± 1.6 (n = 8) |
2.7 ± 4.5 (n = 136) |
1.7 ± 1.5 (n = 92) |
<.001 |
Table 3.
Comparing citations and impact factors between types of open access.
| Measure | Type 1 | Type 2 | Mean citation difference | P value |
|---|---|---|---|---|
| Web of Science citation reports | Gold | Hybrid | −5.5 | .958 |
| Gold | Bronze | −13.4 | .001 | |
| Gold | Green | −14.2 | .003 | |
| Hybrid | Bronze | −7.9 | .888 | |
| Hybrid | Green | −8.7 | .861 | |
| Bronze | Green | −0.8 | .997 | |
| Impact factor | Gold | Hybrid | −0.8 | .859 |
| Gold | Bronze | −2.0 | <.001 | |
| Gold | Green | −0.9 | .049 | |
| Hybrid | Bronze | −1.2 | .667 | |
| Hybrid | Green | −0.1 | .999 | |
| Bronze | Green | 1.0 | .031 | |
Characteristics of OA versus non-OA articles
There was a statistically significant difference in the geographic location of the senior author (P < .001), location of journal publication (P < .001), and journal subspecialty (P < .001) between OA versus non-OA articles (Table 1). Only 26.1% (134/514) of the studies published in North America were OA. In contrast, 59.2% (221/373) of the studies published in Europe and 77.9% (60/77) published in Asia were OA. 23.0% (34/148) of the articles published in pediatric orthopaedic journals were OA. The majority of articles published in general orthopaedic journals were OA (64.1%, 148/231).
There was no difference in study design, type of study, or level of evidence between the OA and non-OA articles (P > .05) (Table 1). However, there were more OA articles published in journals not indexed in JCR (77.2%, 115/149) compared to those indexed in JCR (37.9%, 318/840) (P < .001). Only 1 OA article was published in a journal (Journal of Clinical and Analytical Medicine) that was identified on Beall’s list of Predatory Journals and Publishers [6].
Citation impact of OA versus non-OA articles
The mean citation rate for OA and non-OA manuscripts published in the first half of 2012 was 21.4 ± 28.7 and 18.7 ± 21.4, respectively. This difference in mean citation count between OA and non-OA articles was not statistically significant by univariate analysis (P = .144) (Table 1).
Based on multiple regression analysis, OA publication status was associated with 3 additional citations over a decade (Beta: 3.22, P = .047). Within the context of the mean citation count for non-OA articles over a decade (18.7 ± 21.4), 3 additional citations would represent a 16.0% increase in citations. In this model, papers with senior authors from outside of North America and Europe were associated with a lower number of WoS citations on average (Beta: −5.15, P = 0.003). Studies in journals with higher impact factors, higher (closer to one) levels of evidence, and funded studies were associated with a higher number of citations (Table 4).
Table 4.
Multiple regression with geographical variables coded binarily with Web of Science citation count as the dependent variable.
| Independent variable | Beta coefficient, standard error | 95% CI | P value |
|---|---|---|---|
| Open access | 3.22, 1.62 | 0.04, 6.40 | .047 |
| Level of evidence | −5.67, 1.09 | −7.81, −3.53 | <.001 |
| Impact factor of journal (2012) | 3.01, 0.31 | 2.40, 3.63 | <.001 |
| Multicenter study | 3.45, 2.42 | −1.29, 8.20 | .154 |
| Funded study | 4.34, 2.01 | 0.38, 8.29 | .032 |
| JCR indexed | 4.70, 3.14 | −1.47, 10.87 | .136 |
| Geographic location of senior author—Outside North America/Europe | −5.15, 1.73 | −8.56, −1.75 | .003 |
| Geographic location of journal—Outside North America/Europe | −2.91, 3.16 | −9.10, 3.29 | .357 |
CI, confidence interval; JCR, Journal Citation Reports.
Discussion
The primary aim of this study was to determine if publishing OA confers a citation advantage for pediatric orthopaedic studies. In 2012, the prevalence of OA publications for pediatric orthopaedic studies was 43.8%. Among all orthopaedic journals, the prevalence of OA publications was 41.3%. The distribution of the level of evidence did not differ between OA and non-OA publications. In univariate analysis, there did not appear to be an OACA. However, based on multiple regression analysis, OACA appears to exist in pediatric orthopaedics, with OA status conferring 3 additional citations over 10 years. Studies with funding, higher levels of evidence, senior authors from Europe or North America, and published in journals with higher impact factors were also cited more. Furthermore, there was a statistically significant difference in citation count among the OA types, with Green and Bronze cited significantly more than Gold. The largest difference was that Green OA articles were cited 14.2 more times, as recorded by WoS, than Gold OA articles. One OA article in this study was published in a journal listed in Beall’s List of Predatory Journals and Publishers. JCR indexes well-recognized journals that meet a certain level of quality, and of the 149 articles published in 78 journals not indexed by JCR, 77.2% were OA.
In this study, 43.8% of articles were published OA, similar to rates of 45% to 50% reported in clinical medicine and 45.8% in dentistry literature [16], [17]. However, the OA rate was only 23.0% among the studies published in pediatric orthopaedic journals. Other studies have shown that the OA advantage is greater for the more citable articles [22]. It is important to note that there are other confounding variables that may account for this positive association between OA publication and citation count in this cross-sectional study. For example a statistical model supported the conclusion that authors of higher-quality papers are more likely to choose to publish OA when it is an option to purchase [19]. Similar results between OA types were found in a random sample of 100,000 journal articles from 2009 to 2015; Green OA papers were cited 33% above the world average, while Gold OA articles were cited 17% below the world average [16]. Higher rates of citation counts in Green OA may be driven by publication requirements by organizations like the National Institutes of Health that require articles arising from their funds to be deposited in repositories such as PMC. These funded studies have already been through a rigorous selection process by their respective funding organizations to identify the most robust research plans and, as such, may inherently be more significant and relevant. Therefore, Green OACA may be driven by the quality of its studies. On the other hand, Gold OA articles may be cited less due to some OA journals being smaller and less established [16].
This study has a number of inherent limitations. First, despite a rigorous search strategy, it is possible that all potentially eligible studies were not identified. Second, study quality was only assessed by the level of evidence and impact factor of the journal in which each study was published. Third, although associations were identified, causality cannot be inferred from this analysis. Fourth, only articles from January 1st 2012 to June 30th 2012 were evaluated in this study, which may not be representative of the current patterns in publication. Over the past 10 years, there has been a vast increase in orthopaedic publication volume, as well as a steady increase in the number of OA journals [26], [27]. Specific to pediatric orthopaedic, the Journal of the Pediatric Orthopaedic Society of North America was established in 2019 [28]. These may all impact future publishing practices amongst pediatric orthopaedic surgeons. Fifth, this study evaluated the number of citations only. It is important to consider that cited articles may have been referenced because OA articles were more accessible [29]. It is possible that higher quality or more appropriate citations may exist and that the quality of available citations has changed with increasing OA publications. Assessing the quality or accuracy of the citations was beyond the scope of this study. Sixth, non-English studies were excluded as translation services were not afforded within the scope of this study. The analysis presented here is representative of pediatric orthopaedic publications in English and may not be generalizable to OA versus non-OA publications and citations in other languages. Finally, there may have been confounding variables not accounted for in this analysis; however, the 11 factors studied in this project and 8 variables included in the multiple regression present a robust analysis.
In conclusion, pediatric orthopaedic manuscripts published OA are cited more frequently in the subsequent decade. Many factors may contribute to this association; some funding agencies supporting high-quality studies require OA publishing, while on the other hand, researchers looking for a citation may more easily find articles that do not require a trip to the library or a subscription.
There are many reputable OA orthopaedic journals, including but not limited to JCO, Journal of Bone and Joint Surgery Open Access, Journal of the American Academy of Orthopaedic Surgeons Global Research & Reviews, Journal of the Pediatric Orthopaedic Society of North American (JPOSNA®), and the Orthopaedic Journal of Sports Medicine. JCO and JPOSNA® are the only existing pediatric orthopaedics OA journals. JCO has been OA since its inception in 2007 and requires an APC of approximately $2,000 for nonmembers and $1,000 for European Paediatric Orthopaedic Society members. JPOSNA® has been OA since its inception in 2019 and requires an APC of $1500 for nonmembers and $1070 for Pediatric Orthopaedic Society of North America (POSNA) members. The other 2 pediatric orthopaedic journals, JPO and JPO B, are subscription-based journals that offer the option of publishing OA for an APC of $2,800 to $3,500.
Pediatric orthopaedic surgeons may be weighing the benefits and risks associated with publishing OA. Authors should be cautious in avoiding predatory journals which manipulate the pay-for-publication model to charge excessive prices in exchange for publication with little to no official peer review process. Additionally, the quality and reputation of articles published OA may still carry a stigma of inferiority to those published via the more traditional paywall-protected system. On the other hand, citation counts may positively impact academic promotion, grant success, professional credibility, and personal satisfaction; however, these factors are secondary benefits of OA publishing. OA publishing allows for improved visibility and dissemination of research and, in particular, can improve access to orthopaedic information to clinicians in low-income countries with financial barriers.
Additional links
-
•
JPOSNA®, Volume 4, Issue 3: Trends in Pediatric Orthopaedic Publications by Language
-
•
JPOSNA®, Volume 4, Issue 4: Pediatric Orthopaedic Surgery Research Quality: A Decade of Change at POSNA Annual Meetings from 2011-2020
Consent for publication
The author(s) declare that no patient consent was necessary as no images or identifying information are included in the article.
Funding
This study received no funding.
Author contributions
Erikson Nichols: Writing – review & editing, Methodology, Data curation. Peter Cirrincione: Writing – review & editing, Methodology, Data curation. Jonathan M. Schachne: Methodology, Data curation. Colleen Wixted: Methodology, Data curation. Caitlin Penny: Methodology, Formal analysis. Keza Levine: Writing – review & editing, Project administration. Akshitha Adhiyaman: Writing – review & editing, Project administration. Joseph Nguyen: Writing – review & editing, Formal analysis. Emily Dodwell: Writing – review & editing, Data curation, Conceptualization. Sreetha Sidharthan: Writing – original draft, Formal analysis, Data curation, Conceptualization. Grace Wang: Data curation, Conceptualization.
Declaration of competing interests
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
We would like to thank Bridget Jivanelli MLIS, AHIP, Senior Librarian at the Hospital for Special Surgery, for her assistance with the systematic retrieval of the articles assessed in this manuscript.
Appendix A. Supporting information
Supplementary data associated with this article can be found in the online version at doi:10.1016/j.jposna.2024.100072.
Appendix A. Supplementary material
Supplementary material
.
References
- 1.Björk B.C., Welling P., Laakso M., Majlender P., Hedlund T., Gudnason G. Open access to the scientific journal literature: situation 2009. PLoS One. 2010;5(6) doi: 10.1371/journal.pone.0011273. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Laakso M., Welling P., Bukvova H., Nyman L., Björk B.C., Hedlund T. The development of open access journal publishing from 1993 to 2009. PLoS One. 2011;6:20961. doi: 10.1371/journal.pone.0020961. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Kurata K., Morioka T., Yokoi K., Matsubayashi M. Remarkable growth of open access in the biomedical field: analysis of PubMed articles from 2006 to 2010. PLoS One. 2013;8(5) doi: 10.1371/journal.pone.0060925. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Iyandemye J., Thomas M.P. Low income countries have the highest percentages of open access publication: a systematic computational analysis of the biomedical literature. PLoS One. 2019;14(7) doi: 10.1371/journal.pone.0220229. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Nicholas D., Watkinson A., Volentine R., Allard S., Levine K., Tenopir C., et al. Trust and authority in scholarly communications in the light of the digital transition: setting the scene for a major study. Learn Publ. 2014;27(2):121–134. [Google Scholar]
- 6.Beall, J. Beall's list of predatory journals and publishers.2016. Available at: https://beallslist.net/. Accessed August 1, 2022.
- 7.Beall J. Predatory publishers are corrupting open access. Nature. 2012;489:179. doi: 10.1038/489179a. [DOI] [PubMed] [Google Scholar]
- 8.Yan J.R., Baldawi H., Lex J.R., Simchovich G., Baisi L., Bozzo A., et al. Predatory publishing in orthopaedic research. J Bone Jt Surg Am. 2018;100(21) doi: 10.2106/JBJS.17.01569. [DOI] [PubMed] [Google Scholar]
- 9.Sabharwal S., Patel N., Johal K. Open access publishing: a study of current practices in orthopaedic research. Int Orthop. 2014;38(6):1297–1302. doi: 10.1007/s00264-013-2250-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Doughty K., Rothman L., Johnston L., Le K., Wu J., Howard A., et al. Low-income countries’ orthopaedic information needs: challenges and opportunities. Clin Orthop Relat Res. 2010;468(10):2598–2603. doi: 10.1007/s11999-010-1365-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.SPARC Europe. The Open Access Citation Advantage Service (OACA). Available at: https://sparceurope.org/what-we-do/open-access/sparc-europe-open-access-resources/open-access-citation-advantage-service-oaca/. Accessed August 1, 2022.
- 12.Eysenbach G. Citation advantage of open access articles. PLoS Biol. 2006;4(5) doi: 10.1371/journal.pbio.0040157. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Davis P.M., Lewenstein B.V., Simon D.H., Booth J.G., Connolly M.J., et al. Open access publishing, article downloads, and citations: randomised controlled trial. BMJ. 2008;337:a568. doi: 10.1136/bmj.a568. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Davis P.M. Open access, readership, citations: a randomized controlled trial of scientific journal publishing. FASEB J. 2011;25(7):2129–2134. doi: 10.1096/fj.11-183988. [DOI] [PubMed] [Google Scholar]
- 15.Langham-Putrow A., Bakker C., Riegelman A. Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PLoS One. 2021;16(6) doi: 10.1371/journal.pone.0253129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Piwowar H., Priem J., Larivière V., Alperin J.P., Matthias L., Norlander B., et al. The state of OA: a large-scale analysis of the prevalence and impact of Open Access articles. PeerJ. 2018;6:e4375. doi: 10.7717/peerj.4375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Hua F., Sun H., Walsh T., Worthington H., Glenny A.M. Open access to journal articles in dentistry: prevalence and citation impact. J Dent. 2016;47:41–48. doi: 10.1016/j.jdent.2016.02.005. [DOI] [PubMed] [Google Scholar]
- 18.Björk B.C., Solomon D. Open access versus subscription journals: a comparison of scientific impact. BMC Med. 2012;10:73. doi: 10.1186/1741-7015-10-73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gaulé P., Maystre N. Getting cited: does open access help? Res Policy. 2011;40(10):1332–1338. [Google Scholar]
- 20.Lansingh V.C., Carter M.J. Does open access in ophthalmology affect how articles are subsequently cited in research? Ophthalmology. 2009;116(8):1425–1431. doi: 10.1016/j.ophtha.2008.12.052. [DOI] [PubMed] [Google Scholar]
- 21.Ottaviani J. The post-embargo open access citation advantage: it exists (probably), it’s modest (usually), and the rich get richer (of course) PLoS One. 2016;11(8) doi: 10.1371/journal.pone.0159614. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Gargouri Y., Hajjem C., Lariviére V., Gingras Y., Carr L., Brody T. Self-selected or mandated, open access increases citation impact for higher quality research. PLoS One. 2010;5(10) doi: 10.1371/journal.pone.0013636. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Antelman K. Do open-access articles have a greater research impact? Coll Res Libr. 2004;65(5):372–382. [Google Scholar]
- 24.Zhang D., Blazar P., Kilgallen E.E., Earp B.E. Impact of conventional and open access publications in orthopaedic surgery. J Am Acad Orthop Surg. 2021:e1239–e1245. doi: 10.5435/JAAOS-D-20-01074. [DOI] [PubMed] [Google Scholar]
- 25.Howick J., Chalmers I., Glasziou P., et al. The Oxford Levels of Evidence 2. Oxford Centre for Evidence-Based Medicine. Available at: https://www.cebm.ox.ac.uk/resources/levels-of-evidence/ocebm-levels-of-evidence. 2011. Accessed August 1, 2022.
- 26.Sun J., Mavrogenis A.F., Scarlat M.M. The growth of scientific publications in 2020: a bibliometric analysis based on the number of publications, keywords, and citations in orthopaedic surgery. Int Orthop. 2021;45(8):1905–1910. doi: 10.1007/s00264-021-05171-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Franko O.I., Okike K. Open access and online journals in orthopaedics: what does the future hold? JBJS Rev. 2014;2(7) doi: 10.2106/JBJS.RVW.M.00118. [DOI] [PubMed] [Google Scholar]
- 28.Noonan K. Editor’s note. J Pediatr Orthop Soc North Am. 2023;5(4) doi: 10.55275/JPOSNA-2023-815. [DOI] [Google Scholar]
- 29.Cooke R., Jain N. Open access journals are as likely to be referenced by the Orthopaedic literature, despite having a lower impact factor than subscription-based journals. SICOT J. 2021;7:64. doi: 10.1051/sicotj/2021062. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supplementary material