Open Access Publishing: A Study of Current Practice in Oral and Maxillofacial Surgery Research

Abstract

Introduction

Open access (OA) publication has become an increasingly common route for dissemination of scientific research findings. However, it remains a contentious issue with continued debate as to its impact on the peer-review process and a potential change in the quality of subsequent evidence published. There is little research that looks into OA in oral and maxillofacial surgery.

Methods

We investigated the OA policy in the 30 relevant journals listed in the Institute for Scientific Information Web of Knowledge journal citation report, comparing bibliometric data and quality of evidence produced in journals offering OA and those with subscription-only policies.

Results

3474 articles were graded for evidence level and the results correlated to journal OA status. 76.7 % of journals offered authors OA services. There was no difference between impact factor, self-citation rate, total citations or quality of evidence between OA and subscription journals.

Discussion

These findings should send clear messages to both clinicians and researchers and should re- assure readers that scientific findings that are disseminated in open access form do not differ in quality to those in subscription-only format. It should reinforce that open access formats are a credible way to display research findings in oral and maxillofacial surgery.

Introduction

Open access to scientific literature is freely available on the public internet and allows any reader to use them for any other lawful purpose, including copying, downloading and distributing, without financial, legal, or technical barriers other than those inseparable from gaining access to the internet itself [1]. Over the last 20 years, open access (OA) publication has become an increasingly common route for dissemination of scientific research findings. Between the years 2000 and 2011, the total number of published OA articles rose from 20,700 to 340,000, the number of active OA journals increased from 744 to 6713 and the average number of OA articles published per journal doubled from 26 to 51 [2]. Despite its increasing popularity, OA remains a contentious issue with continued debate as to its impact on the peer-review process and a potential change in the quality of subsequent evidence published.

Traditionally, journals have relied on revenue generated from reader subscription fees to ensure their costs are appropriately accounted for. An OA policy transfers these costs onto the authors or the institution they represent, thus allowing free accessibility for the end reader. Advantages of such a system include cost-free unrestricted access to new evidence, relieving financial pressures both on libraries and individual readers. However the key reservation in OA publishing is that of quality control—whether author-led funding of the publication process may influence the peer review process, the decision to publish and ultimately a decline in the standard of published research [3].

Recent work in other medical fields has demonstrated that there appears to be limited knowledge of OA publishing amongst researchers in general [4]. Despite reservations about the OA model, several studies have supported the notion that the scientific impact and quality of evidence in OA research is not dissimilar from that of traditional subscription journals [5, 6]. To date, there has been little investigation on the practice of OA publishing in oral and maxillofacial surgery (OMFS).

The aim of this study was to provide an overview of OA publishing in OMFS research. The principle goals were to report on the proportion of OMFS journals that publish OA articles, to analyse bibliometric data arising from these journals and to determine whether the quality of evidence they produce is comparable to that of conventional subscription-based journals.

Methods

The Institute for Scientific Information (ISI) Web of Knowledge journal citation report (JCR^®) was used to generate a list of OMFS-related journals that were the focus of this study. The JCR^® directory does not include a specific category for OMFS therefore the categories of ‘dentistry and oral surgery’ and ‘otolaryngology’ were examined. All journals relating to OMFS, including implant dentistry and head and neck surgery, were included for study. Citation data from 2013 was generated within the JCR^®.

Journal websites were used to identify all full-length articles published in 2013. Each journal’s website was also examined to determine its OA publishing policy and publication fees. All abstracts for full-length articles were analysed to identify the level of scientific evidence of each article and graded according to the Oxford Centre for Evidence-based Medicine guidelines [7]. Raters were blinded from the journal’s OA status. Data extracted from each journal included title, region, level of evidence of each article published, total citations, self-citations and journal impact factor (2013). Data was collated using Microsoft Excel (V14.4.1). Because data was non-parametric, the Mann–Whitney U-test was used to compare OA or subscription-only journals. A p value of <0.05 was viewed to indicate statistical significance.

Results

Thirty OMFS related journals were identified as relevant to this study. Fifteen journals originated from USA, 12 from Europe and 3 from Asia (Fig. 1). All article abstracts were published in English. Assessment of bibliometric data from the year 2013 for all the journals in the JCR^® revealed that the median impact factor was 1.28, the median number of total journal citations was 2282 and the median proportion of self-citations was 0.085.

Fig. 1.

The global distribution of OA and subscription-only journals in OMFS

3474 full-length articles were published in 2013. All articles were graded for level of evidence. The median number of level I, II, III, IV and V studies published in 2013 by the selected journals was 4, 6, 19, 28, 35 respectively. The mean level of evidence was 3.84.

Regarding OA publishing, 3 policies were identified. First, there were 7 (23.3 %) journals that did not offer any form of OA publishing to researchers and access to their research was provided by institutional or personal subscription to the journal. Second, there were 20 (66.7 %) journals that were subscription-based which, however, offered researchers the option to publish their work as OA if they paid a specified publication fee. Such journals are often termed ‘hybrid journals’. Lastly, 3 (10 %) journals had an entirely OA policy. Therefore 23 journals (76.7 %) offered some form of OA option. Article processing fees in these journals ranged from $160 to $3800 (median $3000).

For evaluation of statistical differences in bibliometric data, the number of level I studies published and the mean level of evidence, journals that offered OA were compared with subscription only journals. There was no statistical difference in the impact factor (1.27 vs. 1.72), self-citation rate (0.09 vs. 0.08) or total citations (2618 vs. 1395). There was a trend towards greater numbers of level I evidence published in subscription-only journals but this was not significant. There was also no difference in the mean level of evidence (3.92 vs. 3.56). Descriptive data and statistical analysis are summarized in Table 1.

Table 1.

Summary of statistical comparison between OA and subscription-only journals

	Open access	Subscription-only	p value
Median impact factor	1.27	1.72	0.683
Median citations in 2013	2618	1395	0.853
Median proportion of self cites	0.09	0.08	0.437
Median number of level I studies	2	13	0.091
Median number of level II studies	6	10	0.477
Median number of level III studies	20	18	0.787
Median number of level IV studies	26	36	0.883
Median number of level V studies	33	42	0.731
Mean level of evidence	3.92	3.56	0.176

Discussion

This study highlights a number of important issues in regards to OA publication in OMFS. Firstly the process is widespread in journals relating to OMFS with the vast majority (76.7 %) of journals studied offering authors an OA option which includes both solely open access journals (10 %) and “hybrid” journals (66.7 %). The uptake of OA publication in other medical fields appears to fluctuate between specialties at present. A recent study found that in orthopaedic research the proportion of subscription-only, hybrid and solely open access journals was 60.3, 31.7 and 8 % respectively, while a similar study of respiratory journals found that the majority were hybrid journals [8, 9].

A second important finding in this study was that there was no difference in bibliometric data between journals offering OA options and those that did not. Similarly, the average level of evidence was not significantly different. This finding was replicated when looking at the proportion of the different evidence levels published. This importantly suggests that journals offering OA to authors, despite an altered business model, continue to maintain the standards in publication within OMFS.

A third important finding relates to a significant but variable cost to the author, which could have important national and global impacts. While the advantage of OA in low-income institutions or countries, for example, is without doubt, it would also preclude authors and institutions without sufficient financial support from contributing the global evidence base. It should also be noted however, that individual journals may also be limited to the OA strategies of the publishing houses that manage their production, and their control over pricing and even OA policy may be restricted.

Given the assessment of article evidence level was inherently a subjective process, one key strength of this study was its blinded analysis, as assessors were not made aware of the OA policy of the journal in which the article appears. Our judgment of evidence level is consistent with previous studies in OMFS. Analysis of evidence in British Journal of Oral and Maxillofacial Surgery and International Journal of Oral and Maxillofacial Surgery found that 3 % of articles were published as level I evidence, a similarly small percentage as found in our study (5.5 %) [10]. Previous work also found predominance in level 4 and 5 evidence, which is consistent with our data [11].

There are several weaknesses in this study. Differences between both bibliometric data and quality of published evidence in OA and subscription-only journals were identified using the ISI Web of Knowledge JCR^®. The JCR^® is an annual publication by Thomson Reuters, providing citation information for more than 10,000 journals from over 2500 publishers in approximately 232 disciplines from 83 countries [12]. Journals listed within it are by definition well established, PubMed indexed journals with noted publication history. While it is likely that these are journals that the OMFS community will look to as their evidence base, the database is not exhaustive and as such remains a source of selection bias. Indeed as OA becomes more favourable and popular, other databases and directories such as the Directory of Open Access Journals and PubMed Central^® may contain increasing numbers of key studies. However, in contrast, it should also be noted that sham journals have been set up to exploit authors using the open-access model, either publishing work without quality checks or simply duping researchers into paying article processing fees on fraudulent websites [13, 14].

A further problem with the use of the JCR^® was the lack of a specific OMFS category. Our journal list, in an effort to remain as inclusive as possible, was compiled of relevant journals from otolaryngology and dentistry/oral surgery. While our aim to be inclusive of all potentially relevant journals could be seen as a strength, and gave rise to over 3000 abstracts for analysis, individual articles within these journals may not be related to OMFS, again potentially leading to unreliable results. Furthermore, this study looked at OA policy and evidence by journal. Within hybrid journals, there may be a difference between the quality of evidence produced in individual OA articles compared with conventionally submitted ones. Such differences would not have been identified with our study design.

What this study does not attempt to explore is the green and gold nomenclature for open access publishing, which represents the two main routes to open access. Authors making their work available though a “green” OA policy will archive them in either institutional or central repositories such as PubMed Central^®. This form of open access is not subject to article processing charges, although access may be subject to an embargo as per publisher policies. Gold open access models are what are typically thought of as “open access”—namely work is made freely available to the end user via the publishers website. Such publications require article-processing charges and are not subject to any embargo period.

Further work could develop upon these themes. Quality of specific articles within hybrid journals could be assessed and correlated to their own OA status. Sources of funding could be outlined and findings could be linked to “green” and “gold” policies. Other work could look to be inclusive of open access databases to provide a broader analysis of OA literature in the specialty. Although beyond the scope of this study, these represent important avenues for expanding this work.

In conclusion, OA journals are commonplace globally in OMFS research. Over 75 % of OMFS related journals catalogued in the ISI Web of Knowledge JCR^® provide authors with the opportunity to publish their articles as OA for a publication fee. This fee however is highly variable. Comparison of OMFS journals which offer OA with subscription-based journals revealed no significant difference in impact factor, citation count and self-citation proportion in 2013. There was no significant difference in the number of level I, II, III, IV or V studies published between these journal types. Our findings highlight that despite current reservations of OA publication, the standard of evidence published in journals offering OA is no different to that of subscription-only publications. These findings should send clear messages to the OMFS community. Results should re-assure readers that scientific findings that are disseminated in open access form do not differ in quality to those in subscription-only format and should also emphasize to potential authors that open access formats are a credible way to display their research findings.

Compliance with ethical standards

Conflict of interest

The authors (A. Tahim, H. Bansal, A. Goodson, K. Payne or S. Sabharwal) declared that they have no conflicts of interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.