Authors attain comparable or slightly higher rates of citation publishing in an open access journal (CytoJournal) compared to traditional cytopathology journals - A five year (2007-2011) experience

Abstract

Background:

The era of Open Access (OA) publication, a platform which serves to better disseminate scientific knowledge, is upon us, as more OA journals are in existence than ever before. The idea that peer-reviewed OA publication leads to higher rates of citation has been put forth and shown to be true in several publications. This is a significant benefit to authors and is in addition to another relatively less obvious but highly critical component of the OA charter, i.e. retention of the copyright by the authors in the public domain. In this study, we analyzed the citation rates of OA and traditional non-OA publications specifically for authors in the field of cytopathology.

Design:

We compared the citation patterns for authors who had published in both OA and traditional non-OA peer-reviewed, scientific, cytopathology journals. Citations in an OA publication (CytoJournal) were analyzed comparatively with traditional non-OA cytopathology journals (Acta Cytologica, Cancer Cytopathology, Cytopathology, and Diagnostic Cytopathology) using the data from web of science citation analysis site (based on which the impact factors (IF) are calculated). After comparing citations per publication, as well as a time adjusted citation quotient (which takes into account the time since publication), we also analyzed the statistics after excluding the data for meeting abstracts.

Results:

Total 28 authors published 314 publications as articles and meeting abstracts (25 authors after excluding the abstracts). The rate of citation and time adjusted citation quotient were higher for OA in the group where abstracts were included (P < 0.05 for both). The rates were also slightly higher for OA than non-OA when the meeting abstracts were excluded, but the difference was statistically insignificant (P = 0.57 and P = 0.45).

Conclusion

We observed that for the same author, the publications in the OA journal attained a higher rate of citation than the publications in the traditional non-OA journals in the field of cytopathology over a 5 year period (2007-2011). However, this increase was statistically insignificant if the meeting abstracts were excluded from the analysis. Overall, the rates of citation for OA and non-OA were slightly higher to comparable.

INTRODUCTION

It has been more than a decade since the publication of The Budapest Declaration, a landmark article, which was the result of a meeting of key players including many Nobel-laureates from the Open Access (OA) movement[1]. This declaration stated in part, “An old tradition and a new technology have converged to make possible an unprecedented public good.” They were speaking of using the internet and OA principles to disseminate scientific knowledge obtained through research to more people than ever before.[1,2,3,4]

In the time since this declaration, the scientific world has seen a steady increase in the acceptance of the OA publication charter as a robust and viable method of publication, thereby increasing the impact of OA on the scientific literature. This has increased the number of OA publications on the internet, which are available freely to anyone with internet access. Major societies, government agencies, top publishers, and consortiums in the scientific community have followed by publishing many additional declarations supporting the use of OA.[5]

One reason for the growth of OA in the medical community is the known advantage this platform has for both the readers and the authors. In 2001, Steve Lawrence reported in Nature a sentinel publication after analyzing 119,924 articles and concluded that free online availability of scientific publications increased citation rates.[6] Kurtz et al.,[7] Harnad et al.,[8] and others published similar results.[5,9,10]

Other than increasing the citation rates[11], an additional relatively less appreciated beneficial aspect of the OA charter is the retention of copyright by the intellectual property (IP) owner of the individual publication, that is its author/researcher[2,12]. The efforts, time, skills, talent, and many more assets, including variety of public resources, contributed by the ethical owner, the author (s) of the individual publications, are very important and deserve further consideration. Not to lose this IP to any group with restricted benefits to general academia and the public is a major benefit of the OA charter, which is achieved by applying the Creative Commons Attribution License[13], allowing retention of published material in the public domain. Authors are increasingly experiencing the benefits of this feature, which leads to more freedom in sharing and utilizing previously published unique materials such as images, figures, tables, etc., The benefit is applied to numerous academic activities including but not limited to writing reviews, chapters, books, and other teaching material, simply by citing the source of the original information.[14]

Authors have a better chance of becoming a renowned expert on their given subject, by seamless global distribution of their effort to anyone in the world with internet access. An additional advantage gained by authors, readers and the medical community as a whole, and perhaps the most important benefit of OA, is the advances in discovery and treatment as purported by the translational research model, which are made possible by barrier free dissemination of scientific knowledge.

No studies to date have looked at the impact of Open Access publishing on the citation rate in a small subspecialty field like cytopathology, where the majority of journals have been traditional-type publications. Our hypothesis, based on the findings reported previously concerning open access publishing,[5,6,7,8,9,10] is that for the same author publishing in both types of cytopathology journals, the publications in OA Cytopathology Journal such as CytoJournal under the Open Access charter will have a similar or higher citation rate (CR) as compared to the publications in the traditional non-OA cytopathology journals.

MATERIAL AND METHODS

The data in this study was collected solely from the Web of Science based on which impact factors (IF) are calculated.[15] The traditional non-OA journals analyzed are Acta Cytologica, Cancer Cytopathology, Cytopathology, and Diagnostic Cytopathology. This was compared with similar data for the publications in CytoJournal as OA cytopathology journal. The five years, 2007-2011, chosen arbitrarily are closer to the current year of 2013 with reasonable time needed for generation of citations for most of the journals and publications.

The authors selected for this study were those who fit the following criteria:

• Those who published in CytoJournal at least two times within the time period of 2007- 2011

• Those who published in non-open access traditional cytopathology journals (Acta Cytologica, Cancer Cytopathology, Cytopathology, and/or Diagnostic Cytopathology) at least twice within the time period of 2007-2011

• Those who were not past or current editors/co-editors of the journals under study.

Each publication by these authors from the journals selected and the number of citations garnered by each publication as of August 22, 2013 was recorded using the Web of Science citation analysis site [Figure 1].[15] Web of Science was chosen as the database for our study for several reasons. It is a very large database with over 37 million records, and it includes all of the relevant and credible journals in the field of cytopathology. The database is also publisher neutral” giving equal treatment to commercial, OA, societal and university publications.[15] Citations of each publication for each author in both Open Access and traditional non-OA journals were noted and categorized by the publication year. The citations per publication (CPP) for the two journal types were compared.

Figure 1.

Use of the ‘Web of Sciences Database’ to harvest the raw data on number of citations for various publications for different authors publishing in cytopathology journals under study [Table 2]. (a) Add the name of the author and the journal. Click ‘Search’; (b) Manually select the publications during 2007 through 2011. Note the citation numbers and reference details [Table 2]

We designed another value to take into account the influence of the time factor after the publication (i.e. giving more power to publications which have had less time since publication). This metric, the ‘time adjusted citation quotient’ (Q value) was defined and calculated as follows:

N (Yr): Total number of citations for all publications under consideration in a specific year (Yr).

C (Yr): Average number of CPP in a specific year (Yr) by all cytopathology journals under study.

X: Total number of publications under consideration for that author from 2007 to 2011 for that journal category (OA or non-OA).

The Web of Science's citation analysis [Figure 2][15] was the source of the average number of CPP for that year in that journal i.e. C (2007), C (2008), C (2009), C (2010), and C (2011). Upon completion of the search for each of the cytopathology journal, C (Yr), the mean value for each year was calculated [Table 1]. The Q value for each publication in CytoJournal as OA cytopathology journal as well as the traditional non-OA cytopathology journals was then calculated by above formula. Minitab software[16] was used for statistical analysis.

Figure 2.

Use of ‘Web of Sciences citation analysis tool’ for finding the average number of citations per publication [C (Yr)] in a specific year for all the cytopathology journals under study. (a) Add the name of the journal; (b) Click ‘publication years’, choose the specific year of interest (2008); (c) Click ‘Create Citation Report’ link on the next screen; (d) Note the ‘Average Citations per Item’ C (Yr) for that journal for that year (2008) [Table 1]

Table 1.

Average number of citations per publication in a specific year [C (Yr)] for various cytopathology journals

To compare CRs for CytoJournal as an OA cytopathology journal versus non-OA cytopathology journals, by using freely available data on the web, a few of these authors were also analyzed arbitrarily by ‘Publish or Perish’ software which uses ‘Google scholar’ data.[17,18]

RESULTS

A total of 28 authors were identified as per the criteria who published papers or meeting abstracts in both OA and non-OA journals. When meeting abstracts were excluded 25 authors were considered. Overall, a total of 314 publications in cytopathology journals during 2007-2011 were evaluated based on the data from web of science citation analysis site (Impact factor is calculated based on this data) [Table 2].[19–325] Some publications were attributed to more than one author included in the study. The data shown in red in Table 2 indicate publications as meeting abstracts. Because OA cytopathology journal and some non-OA cytopathology journals did not publish meeting abstracts on regular basis and this feature may potentially impact the final comparison, we analyzed the data in two ways: First on all the data including the meeting abstracts, and then repeating the analysis after excluding the meeting abstracts.

Table 2.

Authors and citations (raw data)

In the group in which meeting abstracts were included, the combined number of publications per author ranged from 4 to 41 with an average of 15.6 and a median of 15. The number of publications per author in CytoJournal ranged from 2 to 9 with an average of 3.1 and a median of 2. The number of publications per author in traditional non-OA journals ranged from 2 to 32 with an average of 12.5 and a median of 15.

In the group in which meeting abstracts were excluded, the combined number of publications per author ranged from 3 to 27 with an average of 11.6 and a median of 10. The number of publications per author in CytoJournal ranged from 2 to 9 with an average of 3.1 and a median of 2. The number of publications per author in traditional non-OA journals ranged from 1 to 25 with an average of 8.4 and a median of 7.

The citations per publication (CPP) and the time adjusted quotions (Q values) were calculated for CytoJournal as an OA journal versus the traditional non-OA cytopathology Journals with the meetings abstracts included [Table 3]. Overall, the averages of both CPP and Q values were higher for OA Cytopathology Journal (cytojournal) than the traditional non-OA journals. To confirm our hypothesis, paired t-tests were run on both data sets (CPP and Q values) using Minitab software.[16]

Table 3.

Comparison of ‘Citations per publication’ and ‘time adjusted citation quotients’ (Q values) for CytoJournal as OA cytopathology journal versus non-OA cytopathology journals with meeting abstracts included [Figure 3]

For CPP, the following methodology was used. To make sure valid paired t-tests could be used, an Anderson-Darling normality test was run on the differences between CPP for CytoJournal as OA cytopathology journal and for the traditional non-OA journals. The normality test was passed and the paired t-test was run. A 95% confidence interval for the mean difference between [CytoJournal citations per publication] – [traditional non-OA citations per publication] was generated with the interval being (1.406, 3.824) with a P value of 0.001.

The same methodology was used to analyze the Q values. The 95% confidence interval for the mean difference between [CytoJournal Q value] - [traditional non-OA Q value] was (0.473, 1.084) with a P value of approximately 0.0001. The findings confirmed the hypothesis that the publications in OA cytopathology journal generated improved citation rate (CR) with higher CPP and Q values with statistically significant difference as compared to the publications in the traditional non-OA cytopathology journals [Figure 3a and b].

Figure 3.

With the second set of data, without the inclusion of meeting abstracts [Table 4], the same paired t-tests were run, with the null and alternative hypothesis similar to that for the first set of data. CPP with Open Access versus Non-Open Access showed a 95% confidence interval of (-1.038, 1.848) with a P value of 0.568. For the Q value, a 95% CI interval of (-0.309, 0.677) with a P value of 0.448. This analysis showed that CPP and Q values were also higher when meeting abstracts were taken out of the data set, but the difference was statistically insignificant [Figure 3c and d].

Table 4.

Comparison of ‘Citations per publication’ and ‘time adjusted citation quotients’ (Q values) for CytoJournal as OA cytopathology journal versus non-OA cytopathology journals without meeting abstracts [Figure 3c and d]

The results with ‘Publish or Perish’ software using ‘Google scholar’ data[17,18] also showed comparable pattern with higher citation rates for the publications in OA cytopathology journal than the traditional non-OA cytopathology journals. This data was unfiltered and included citations by all sorts of publication types.

DISCUSSION

A citation is defined as “a quoting of an authoritative source for substantiation”.[11] As almost all authors would like to be seen as “authoritative source” and their work as “substantial,” citations are a crucial metric in determining the success of both authors and journals. They are used in calculating such relied upon publication metrics in journology as impact factor and H-factor, which are used critically by many in evaluating the worthiness of a journal or an author[12]. Citations are indexed in several large databases on the World Wide Web, the largest of which is Thomson Scientific's Web of Science® which currently contains more than 40 million bibliographic records and 550 million citations from the past 100 years. We conducted the current study using the same data which is also used to calculate impact factor (IF)[15].

Since the Budapest declaration, several studies have examined the impact of the Open Access model of publication on the rates of citation for publications/authors. In 2001 in Nature, Steve Lawrence was the first to publish that free online availability of a publication greatly increased its impact on the scientific community. He analyzed CRs for 119,924 conference articles in computer science and related disciplines and excluded self-citations. He demonstrated the relationship of online availability as a function of the number of citations per article and the year of publication. The results were quite dramatic, showing a direct relationship between the factors, specifically a 157% increase in citations for articles that were free online compared to those which were not available free online.[6]

Our study showed that, in the field of cytopathology, authors who published in both OA cytopathology journal and traditional non-OA journals, accrued a relatively higher rate of citation per publication and time adjusted citation quotient for their publications in the OA journal with statistical difference (P < 0.01) [Table 3 and Figure 3a, b]. However, if meeting abstracts were excluded from the analysis, increase in CPP and Q values was statistically insignificant [Table 4 and Figure 3c, d] This data supports the prior published conclusions that the OA model is a legitimate platform for publication with comparable or even higher citation rates to traditional journals.

Kurtz et al. studied the increased CRs in OA publications but noted a possibility of selection bias. The suggested bias was that the most prominent authors are more likely to make their publications available in an OA model, artificially increasing the rate of citations.[7] As previously mentioned, cytopathology is a uniquely concise field, in which there are very limited numbers of Open Access journals, CytoJournal being one, but several traditional non-OA cytopathology journals. Because of this there are some authors who have publications in both OA and traditional journals, making comparison of the CRs possible while eliminating the above mentioned selection bias suggested by Kurtz et al. as a confounding factor.[7]

Another type of variable is the ‘early view bias,’ wherein a publication that is posted on an OA platform before final publication will have more time to accrue citations and thus skew the data towards citations in OA.[10] The OA journal used in our project (CytoJournal) does not post in pre-publication form, and thus our study was controlled against this type of bias.

Kurtz et al. also discussed another type of bias for which we were not able to control and that might influence the CRs of OA publications. This is a different type of selection bias, wherein the individual author selects their most important (and thus citable) publications for OA.[7] This type of bias is extremely subjective and difficult to prove or refute in a controlled study. However, it is important to highlight that the quality of the published material is generally the primary factor responsible for its overall impact and CR. Traditionally, the quality of published material is predominantly facilitated by the peer-review component of the editorial activity of the peer-reviewed journals. Thus, it is critical to understand and consider the quality of the peer-review process of any scientific journal irrespective of its OA status.

Some concerns have been raised in recent years regarding sprawling, low-quality journals (including some OA journals) which may have high turnover of many publications that have little relevance or contribution to today's scientific discoveries. This is an issue not applicable only to OA, but also to any journal irrespective of its status as an OA journal or traditional non-OA journal. The very core of any reputable scientific journal, with a quality-minded editorial board, is the high standard of the publications received and accepted after a vigorous peer review process with proactive participation by peer-reviewers.

We also evaluated, using a small cohort, the citation pattern of OA versus non-OA cytopathology publications with ‘Publish or Perish,’ software which uses ‘Google Scholar’ data on open, freely available platform.[17,18] As compared to the Web of Science, the inclusion of citations by Goggle Scholar is wider and includes many journals and other platforms which may cite the original work with a relatively liberal approach. The initial analysis with ‘Publish or Perish’ based on ‘Google Scholar’ data showed comparable results with relatively higher rates of citation for CytoJournal practicing OA publication model as compared to traditional non-OA cytopathology journals (without statistical significance).[332]

In summary, this study demonstrated that in the small subspecialty field of cytopathology, authors who published in both an Open Access journal (CytoJournal) and at least one traditional journal (Acta Cytologica, Cancer Cytopathology, Cytopathology, and/or Diagnostic Cytopathology) accrued a comparable or slightly higher CR for OA publications as compared to the traditional non-OA cytopathology journals over a five year period from 2007-2011 [Figure 3].

COMPETING INTEREST STATEMENT BY ALL AUTHORS

No competing interest to declare by any of the authors.

AUTHORSHIP STATEMENT BY ALL AUTHORS

All authors declare that we qualify for authorship as defined by ICMJE http://www.icmje.org/#author.

ETHICS STATEMENT BY ALL AUTHORS

This study did not require approval from Institutional Review Board (IRB) (or its equivalent) as it is based on analysis of published data on web.

EDITORIAL/PEER-REVIEW STATEMENT

CytoJournal editorial team thanks the Academic editor: LaVentra E. Danquah, MIS, MLIS, (laventra@wayne. edu) Coordinator for Instruction, Liaison, and Outreach Services, Shiffman Medical Library, Mazurek Medical Education Commons, Wayne State University, 320 E. Canfield, Detroit, MI 48201, (313.577.9083) for completing the peer-review process for the manuscript of this article.