Open Access Publishing and Subsequent Citations Among Articles in Major Cardiovascular Journals

Abstract

Background:

While open access publishing among cardiovascular journals has increased in scope over the last decade, the relationship between open access and article citation volume remains unclear.

Methods:

We evaluated the association between open access publishing and citation number in 2017 among 4 major cardiovascular journals. Articles indexed to PubMed with ≥5 citations were identified among the following journals: Circulation, European Heart Journal, Journal of the American College of Cardiology, and JAMA Cardiology. Multivariable Poisson regression models were adjusted for journal and article type.

Results:

Of the 916 articles published in 2017, original investigations accounted for most articles (66.7%), followed by reviews (14.5%), guideline/scientific statements (8.4%), research letters (3.7%), viewpoints (3.7%), and editorials (2.9%). Among all articles, 43% (n=391) were open access. Citation number was higher among open access articles compared with those with subscription access (14 [25^th-75^th percentile: 9-23] vs. 11 [25^th-75^th percentile: 7-17]; P<0.001). Open access status was significantly associated with higher number of citations after multivariable adjustment (β coefficient: +0.42, 95% CI: 0.38-0.45, P<0.001). Open access articles had consistently higher citations compared with subscription access articles across the 3 most frequent article types.

Conclusion:

Among contemporary articles published in major cardiovascular journals, open access publishing account for over 40% of articles and was significantly associated with increased short-term citations. Further research is required to assess the variation in long-term citation rates based on open access publishing status.

Open access, an avenue for widespread dissemination of scientific output, has become increasingly common across the medical literature. Open access articles expanded 16-fold within biomedical journals over an 11-year timespan, and may continue to grow as funding agencies mandate unrestricted public access to research.^{1, 2} While open access provides increased availability of scientific output, its relationship to article citation volume remains unclear.^{3, 4} We evaluated the association between open access status and citation number among articles published in major cardiovascular journals in 2017.

We utilized Dimensions, a searchable digital research infrastructure that aggregates data from over 97 million publications,⁵ to identify articles indexed on PubMed with ≥5 citations published in 2017 among 4 major cardiovascular journals (Circulation, European Heart Journal, Journal of the American College of Cardiology, and JAMA Cardiology). Information regarding publishing status and number of citations was available through Dimensions. Abstracts and full-texts were reviewed to extract data regarding article type, article topic, and study design.

The association between open access status and number of citations was assessed using multivariable Poisson regression models adjusting for journal and article type. To limit potential for undue influence of guidelines/scientific statements, sensitivity analysis was performed excluding this article type. Analyses were carried out using R version 3.5.0 (R Foundation for Statistical Computing).

There were 916 articles published in 2017 with 5 or more citations at the time of search query (August 13, 2018). Original investigations accounted for most articles (66.7%), followed by reviews (14.5%), guideline/scientific statements (8.4%), research letters (3.7%), viewpoints (3.7%), and editorials (2.9%). Observational studies (76.1%) were the most common design among original investigations/research letters. The most frequent article topics were: coronary artery disease (23%), heart failure/cardiomyopathy (19%), electrophysiology (15%), and valve disease (9%).

Overall median number of citations was 12 (25^th-75^th percentile: 8 - 20) and 43% (n=391) were open access. Median number of citations was higher among open access articles compared with those with subscription access (14 [25^th-75^th percentile: 9-23] vs. 11 [25^th-75^th percentile: 7-17]; P<0.001). Open access articles had consistently higher citations compared with subscription access articles across the 3 most frequent article types (original investigations, reviews, guidelines/scientific statements) (Figure 1). Open access status was significantly associated with higher number of citations after adjusting for journal and article type (β coefficient: +0.42, 95% CI: 0.38-0.45, P<0.001) (Figure 1). Comparable association with citation volume was observed in sensitivity analysis excluding guidelines/scientific statements (adjusted β-coefficient +0.31, 95% CI: 0.28-0.35, P<0.001). Compared with original investigations, research letters, editorials, and viewpoints were associated with fewer citations, while guideline documents/scientific statements were associated with higher citations (adjusted P<0.001 for all comparisons). There was no significant difference in citation number between review articles compared with original investigations (adjusted β coefficient: −0.03, 95% CI: −0.08-0.02, P=0.19). Goodness-of-fit testing revealed significant residual deviance of the model (residual deviance: 15984, degrees of freedom: 906, P<0.001).

Figure 1.

Association between Open Access Status and Number of Citations. Each data point represents a single article. Citation number was highly skewed and was natural log-transformed for display purposes and analyzed using multivariable Poisson regression (adjusting for journal and article type). Number of articles and citations by publishing status and article type are displayed below the figure.

We described the relationship between open access status and subsequent citations among articles published in 2017 in 4 major cardiovascular journals. Open access articles accounted for over 40% of articles and were more commonly original investigations. Open access status was independently associated with higher short-term citation number after adjusting for article characteristics. In aggregate, citation rate was 1.5 times higher for open access articles compared with those without open access.

Given the costs of open access publication and the importance of citations to journals, academic institutions, and individual researchers, there is interest in understanding the relationship between these 2 entities. Studies have previously evaluated the impact of open access status on citation rate in other fields of medicine, yielding variable findings. A trial within 11 physiology journals that randomized articles to open or subscription access revealed that while open access status resulted in increased readership, there was no observed effect on overall citation rate.⁴ It is unclear whether the findings of this trial extend to the current academic climate, in which open access publishing has continued to expand, or to cardiovascular publications specifically.

Our study has limitations. Despite multivariable accounting, our model showed overall poor fit with observed citation rate. Unmeasured factors likely explain the high degree of residual deviance of our model. Specifically, it is likely that overall scientific merit of articles may be a primary unmeasured confounder and cannot be captured through traditional metrics. Funding source, which may have a role in both publishing status and citation number, was not readily captured through the Dimensions dataset at the time of search query. Indeed, since the establishment of the of the National Institute of Health (NIH) public access policy, more than 100,000 NIH-funded manuscripts are deposited to PubMed Central annually.⁶ As such work has previously undergone a rigorous grant-level peer review process, articles generated from government or societal funding that require open access may be of increased interest to the academic community, and thus may result in higher citation rates. Due to the relative recency of article publication, there may not have been sufficient time for accrual of citations. Finally, it is possible that these journals utilized mechanisms of open access that were not identified by Dimensions.

Among contemporary articles in 4 major cardiovascular journals, open access status is independently associated with higher short-term citation rates. Future efforts are needed to better understand variation in citation patterns and to continue to expand and make affordable open access publishing platforms as a means to disseminate scientific information.

Clinical Significance.

• Open access publishing has expanded over the last decade as several funding agencies have mandated unrestricted access to scientific output.

• Among contemporary articles published in major cardiovascular journals, open access publishing is common, accounting for over 40% of articles.

• Open access publishing is significantly associated with increased short-term citation number, independent of article type and journal of publication.