The Inherent Drawbacks of the Pressure to Publish in Health Sciences: Good or Bad Science

Version Changes

Revised. Amendments from Version 1

We have published a new version, to correct an error in the corresponding author's name.

Abstract

In recent years, there has been a significant increase in the number of scientific publications– it is the era of “hunting the article”. This commentary discusses the drawbacks of the pressure to publish that certainly contribute to the ‘dark side’ of science. In fact, health science career progression greatly relies on the number of scientific publications a researcher has, and in many cases these may be more valorized than the health services provided. Of course, scientific publications help to develop the skills of health care professionals, but as Einstein highlighted “ not everything that counts can be counted, and not everything that can be counted counts”.

Introduction

In recent years, a significant increase in the number of scientific health publications has been registered. Some possible explanations include: (a) the feeling that nowadays “there is no science without being published”, as it corresponds to the permanent record of our research, reputation and “immortality”; (b) the author’s “motivation” to publish due to their need to obtain funding and further their career; (c) the perception that to publish is the individual or team effort to ensure the wide sharing of results; on the other hand, to not publish may suggest that the author is not committed to sharing knowledge and, in some cases, wishes to avoid scientific discussion with peers.

Perhaps in the past it was more difficult to publish than it is at the present. Database access to publications was scarce or nonexistent, and the cycle of publication was time consuming without an online platform. Nevertheless, the competition to publish was not as aggressive, the impact factor and the h index were not a concern, and scientists were not constantly scrutinized according to their publication records/numbers. Nowadays, these aspects have become new worries, increased by new ethical and conflict of interest issues. Thus, to publish is both almost compulsory (a “question of survival”) and simultaneously a very hard task; we are in the era of “hunting the article”, which in some cases may promote fraud and corruption.

How could we explain the 1.7 to 1.8 million articles in 2013 in journals with peer-review (a supposed certificate of quality), or approximately one article every 18 seconds? ¹ Moreover, and in spite of significant innovation, some studies suggest that the average peer-review takes approximately 10.9 and 6.5 hours for young versus experienced reviewers, respectively ². In addition, according to Mabe ³ the number of new peer reviewed journals and articles published annually has been growing at a very steady rate of about 3–3.5% per year for over three centuries, but in the last few years this increase has become more pronounced.

Publishing is also a business and some authors even suggest that it is becoming pathological, with psychological and legal implications ⁴. Indeed, the area of scientific publication moves many materials and human resources. The International Association of Scientific, Technical and Medical (STM) Publishers that holds 66% of the publication market and each year publishes nearly two-thirds of all journal articles, handled $9.4 billion in 2011 (up from $8 billion comparatively in 2008) solely in scientific journals, and employs about 110,000 people. Although the United States of America (USA) continues to dominate the global production of research papers, significant growth has also been registered for China and East Asia.

This commentary aims to highlight some relevant aspects of scientific publications that all health care providers and researchers, as well as medical students, should better understand in order to avoid publication misconduct.

Concerns in selecting scientific journals

In traditional journals, articles are usually behind a ‘pay-wall’, meaning that readers must have a subscription or pay a fee to read content. In opposition to this traditional method, the number of open access (OA) journals have been increasing in the last few years ⁵. OA journal models comprise two main approaches:

1. OA publishing or the “Gold route”

This model of OA publishing has been increasing in popularity ⁶ and represents direct publication in OA journals, with the majority requiring payment of the cost of publication. Nevertheless, some are sponsored, which means that the author does not pay or some institutions may have an agreement with publishers ( i.e. the OA membership). A hybrid or optional variant exists ( i.e. only part of the article is immediately OA and, if wanted, the author may pay for the full article to be OA);

2. Delayed free access and self-archiving, or the “Green route”

In this case, some traditional journals, after an embargo (necessary to recover the investment by publishers), allow the publication of some articles in free repositories ( e.g. MEDLINE, PsyDok). For free access after the embargo these journals may require the payment of article processing charges.

There are around 10,128 fully OA journals listed on the Directory of Open Access Journals ⁷. This publication model is important for future citation and therefore to maximize the impact of research. The clients of the publishers are the authors and not the readers, but the exaggeration in the number of journals and articles may even disquiet the best proponents of the OA model. For example, in 2013 PLOS ONE published approximately 31,500 articles (a staggering increase from 138 articles in 2006), meaning almost one in every 60 PubMed articles if from PLOS ONE ⁸. This “megajournal” structure is widely accepted as one of the most trusted OA journals, practicing a very broad scope and a rapid “non-selective” peer review based on “soundness not significance” ( i.e. selecting papers on the basis that science is soundly conducted rather than more subjective criteria of impact, significance or relevance to a particular community) ². Currently, Brazil is the country with the most OA journals after the USA; more than one thousand. That abundance was recently described as “a plague of Brazilian science: articles of second class” ⁹.

In our opinion one of the best papers alerting the community to this problem was published by Bohannon ¹⁰. The author investigates the peer-review process of the fee-charging OA journals. Between January and August 2013, Bohannon submitted obvious false articles to 304 scientific journals. In the author’s opinion the article should have been immediately rejected by the editors and reviewers, but 60% of journals accepted it. The article was based on multiple combinations of the “X molecule of species Y lichens that inhibits the growth of cancer cells Z”. For each article, Bohannon even faked the authors and affiliations. All articles essentially concluded that those molecules had therapeutic activity in several cancer types. The study also showed a map of the geographical location of the editors and publishers and their bank accounts. The author concluded that any reviewer with basic knowledge of chemistry and with skills to analyze an elementary graph should have immediately detected the article’s faults. He concluded that “it was easy to jump directly from the test tube to the clinic”.

Subsequently, Hvistendahl ¹¹ reported that in China, a black market has been developed by some agencies to allow authors to write articles without a need to perform experimental work. The prices depend on whether the person paying wishes to be listed as the primary writer, or as merely a co-author, or even as just one of the team members. Authors suggested that this black market may have contributed to China’s rapid growth in the number of Science Citation Index (SCI) articles, with China’s contribution now the second largest, after the USA. Curiously, a running joke in China offered another meaning for SCI: “Stupid Chinese Idea” ¹².

Furthermore, researchers are bothered with several invitations to review or submit articles that are not within their main area of research, and the same invitations are distributed to many email accounts. Perhaps also sharing the same annoyance at this state of affairs, Jeffrey Beall in August 2012 published the first edition of the “Criteria for Determining Predatory Open-Access Publishers”. These criteria were the start point to generate a list of potential, possible, or probable predatory OA scholarly publishers.

Similar problems may also occur with traditional journals, even peer-reviewed. The question is no longer if we should have OA journals. In fact, they are a very positive way to share scientific knowledge for all researchers, and many possess great credibility. The question now is what journals we should choose and how to assess their credibility. To help to solve this problem, it is now possible for readers to perform a continuous re-reviewing of published articles as it occurs for F1000Research. Through the readers’ opinions, journal credibility will be scrutinized.

Constraints of the peer review cycle

Peer review is a methodological evaluation on the soundness of the topic, originality, methodology, results and conclusions highlighted by the author and the authorities cited. Although it cannot generally assure that the data is truthful or not, peer review unquestionably increases the quality of most manuscripts. Nevertheless, this procedure is sometimes slow, expensive, profligate, subjective, prone to bias, poor at detecting gross defects, and almost useless in detecting fraud ¹³.

Typically pre- ( e.g. double and single-blind, and open) or post-publication peer review is practiced. Double-blind peer review is more common in the humanities and social sciences than in the exact sciences. Although the identity of the reviewers is not disclosed and vice-versa (which removes the potential influence of the involved countries, institutions or eventual personal conflicts), the process sometimes fails, because the method, style of writing, acknowledgments and abuse of self-citations can suggest the source. In single-blind peer review the author is unaware of the identity of the reviewers; it is the most common process, especially in life sciences and is useful for the reviewer in order to consult previous authors’ works. By the same reasons listed for the double-blind peer-review, the process may sometimes fail. Although less common than the previous peer reviews, the open peer review process is increasing: nevertheless, it results in less acceptances to review since reviewers are afraid of being identified if commenting negatively on the article ¹⁴. Nevertheless, it is expected that reviewers produce better revisions and avoid offensive or rude words. The name of the reviewers and comments may be published alongside the article as it occurs in the British Medical Journal.

Once the pre-publication peer review process is completed, the decision to publish is made by the journal editor on the advice of the reviewers. The editor of a journal is usually an independent, leading expert in his/her field appointed and sometimes financially supported by the publisher. An apparent misuse of editorial privileges was practiced by the editor of “Chaos, Solitons and Fractals” (a theoretical-physics journal), who was criticized by using its pages to publish numerous articles written by himself ( e.g. 36 papers in the December 2008 issue) ¹⁵.

Finally, in post-publication peer review, the quality is assessed by the sapient of crowds and readers can also judge the quality of the review process. This peer review is outside the editor’s monopoly and journals usually also provide a discussion forum about the article in order for researchers to comment on and read other’s comments. Similar to the British Medical Journal, referee reports and names are published alongside the article, together with the authors' responses to raised points. It is believed that both invited and un-invited ( i.e., commenting) post publication peer review helps to increase the quality of the final publication and it is certainly more transparent. F1000Research follows this model and advocates that time is not lost in reviews, since each article is only subjected to editorial verification and within approximately one week is published online already formatted and can be read and cited ( http://f1000research.com/about).

Limitations of the impact factor

In 1960, the Institute for Scientific Information (ISI) was founded by Eugene Garfield and later he proposed the impact factor (IF) as a tool for journal evaluation by librarians to help them with journal purchasing decisions ¹⁶. The ISI was purchased by Thomson Scientific & Healthcare in 1992 becoming the “Thomson ISI”. IF it is not a perfect tool to measure the prestige of the journal, but there is no better. Falagas and colleagues performed a very interesting discussion on metrics for scientific journals as well as for researchers ¹⁷. Some biases of the IF include: (a) it is not statistically representative of individual articles; (b) review articles (often highly cited) greatly skew the results; (c) extensive articles always have many citations and yield a high IF; (d) it includes self-citations; (e) books do not count for citations; (f) databases primarily index articles in English (but the IF also exists for non-English journals) and are dominated by USA publications; (g) it is dynamic since it depends on fluctuations of research in a given area; (h) paid access of some journals; (i) journals with tight scope tend to have low IF; (j) it does not take into account the subjects variability ( e.g., immunology and cancer are usually highly cited); (k) editors aware of the importance of the IF tend to accept articles that may be highly cited and to reduce the number of articles accepted; (l) the absence of an IF in a given journal can result in low submissions; (m) it is only applied to ISI journals; (n) one highly cited article can boost the IF in a given journal ( e.g., the impressive IF of 49.9 for Acta Crystallographica Section A: Foundations of Crystallography; the primary cause of this high impact factor was a single feature article by Sheldrick ¹⁸).

Fraud in life sciences

Nowadays, there is rising interest in research and publication ethics. Proof of that is the increased importance of organizations such as the Committee on Publication Ethics (COPE) and the development of software to detect plagiarism. Although, the number of journal article retractions has grown in the last decade ¹⁹, it is the general consensus that this may be the result of increased awareness rather than misconduct ². Nevertheless, several fraudulent/misconduct cases have been made publicly available.

Andrew Jeremy Wakefield, a former British surgeon and researcher, published a fraudulent study in 1998 claiming that there was an association between the administration of the measles, mumps and rubella vaccine, and the development of autism and Crohn’s disease ^{20, 21}.

The German physicist Jan Hendrik Schön, starred a scandal related to semiconductors that triggered a series of retractions, six of them out of Science ²². Hwang Woo-suk ( i.e. the pride of South Korea) was sentenced to two years in prison with suspended sentence after distorting the results published in two Science articles related to cloning of human embryonic stem cells ^{23, 24}.

More recently in January 2014, Haruko Obokata, a young researcher in Japan published in Nature, showed stem cells can now be made quickly just by dipping blood cells into acid ^{25, 26}. On June 2014, Obokata agreed to retract both the papers and 2 months later Obokata’s mentor and co-author, Yoshiki Sasai, committed suicide by hanging. Although investigation cleared him of misconduct, he was not free of critiques for inadequate supervision of Haruko Obokata.

In 2005, the researchers David Mazières and Eddie Kohler designed an anecdote manuscript, to send in response to unsolicited congress invitations. Later in 2014, Peter Vamplew, an associate professor at the Federation University Australia School of Engineering and Information Technology, after receiving a spam email from the International Journal of Advanced Computer Technology (classified as predatory OA on Beale’s list’), forwarded Mazières’ and Kohler’s old paper as a response. The journal’s peer-review process classified the manuscript as “excellent” and accepted it for publication ²⁷. At the end, the manuscript was not actually published since Vamplew declined to pay the article processing charge. Acceptance of a paper consisting entirely of 863 repetitions of “Get me off your fucking mailing list” has led commenters to question whether the enterprise is more interested in collecting publication fees than in contributing first-rate peer reviewed articles to the advancement of computer science ²⁷.

Some epic/record examples of fraud were practiced by Joachim Boldt and Yoshitaka Fujii. Joachim Boldt is a German anesthesiologist who was dismissed of his professorship and is under criminal investigation for having allegedly faked of up to 90 research studies ²⁸. Yoshitaka Fujii is a Japanese researcher in anesthesiology, who in 2012 was found to have fabricated data in at least 172 scientific papers over the past 19 years, setting what is believed to be a record for the number of papers by a single author requiring retractions ^{29, 30}. Many of the listed co-authors did not know they were authors and their signatures had been forged in the copyright transfer.

Concluding remarks

Being a scientist is a stimulating and gratifying task, but full of difficulties. Scientists do not have schedules and need to fight for financial support; their careers grow exponentially when they publish (especially in high IF journals) and their job security frequently depends on the number of publications. But first of all, scientists are human beings, with families, needs and emotions. Therefore, the motivation to be successful and remain employed may increase the risk of involvement in scientific fraud or even corruption. This may lead authors, journals and publishers to lose their credibility. Particularly problematic is the impact of fraud in areas that have significant impact on the health, safety and welfare of the world’s population, as are the cases of life and health sciences. Moreover, scientific research slows since it is necessary to spend more time confirming published results. Taradi and colleagues show that over 90% of the medical students of Croatia admitted to engaging in education dishonesty and over 78% engaging in academic misconduct ³¹. Indeed, fraud can help the scientist rise, but also fall rapidly.

We need to change the paradigm of scientific research. We cannot grow at all costs. It is also relevant to make peer review more transparent to produce publications that are more genuine and free from bias. The increase in publications, research studies split into multiple publications (rather than single, longer articles), the proliferation of journals, the ways in which academic promotion fosters this proliferation of publications, and the ways in which these changes can encourage bad or even fraudulent science are important topics that will certainly dictate the future of life and health sciences research.

Fortunately, fraud and corruption are punctual cases. Nevertheless, we will face problematic times if financial issues superimpose the ethics in publishing. This “crisis” may be an opportunity and challenge to reflect on these topics.

Funding Statement

Ricardo Dinis-Oliveira acknowledges Fundação para a Ciência e a Tecnologia (FCT) for his Investigator Grant (IF/01147/2013).

I confirm that the funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.