Retracted Science and the Retraction Index

Abstract

Articles may be retracted when their findings are no longer considered trustworthy due to scientific misconduct or error, they plagiarize previously published work, or they are found to violate ethical guidelines. Using a novel measure that we call the “retraction index,” we found that the frequency of retraction varies among journals and shows a strong correlation with the journal impact factor. Although retractions are relatively rare, the retraction process is essential for correcting the literature and maintaining trust in the scientific process.

EDITORIAL

“A man who has committed a mistake, and doesn't correct it, is committing another mistake.”

—attributed to Confucius

Of more than 28,000 articles in its 40-year history, Infection and Immunity has issued only 15 retractions. Six of these were issued this year and arose from a single laboratory (52–55, 87, 89). This has prompted us to reflect on the process of manuscript retraction and its importance for science and to add to our essay series commenting on the descriptors and qualifiers of present-day science (13–16, 27, 28).

Reasons for retraction.

Eight of the articles retracted by Infection and Immunity, including the six most recent instances, were found to contain digital figures that had been inappropriately manipulated (51–55, 78, 87, 89). Six of the others were retracted by the authors after they determined their previously reported findings to be unreliable: two were unable to confirm their original results (42, 67), one discovered that a cDNA library was actually obtained from another organism (38), and three found a critical reagent to be impure (19, 49, 61). The remaining article was retracted due to extensive plagiarism (43). This is a reasonably representative sample of the reasons for manuscript retraction discussed in guidelines from the Committee on Publication Ethics (COPE) (93, 94). A COPE survey of Medline retractions from 1988 to 2004 found 40% of retracted articles to be attributed to honest error or nonreplicable findings, 28% to research misconduct, 17% to redundant publication, and 15% to other or unstated reasons. Research misconduct is classified as falsification or fabrication, with falsification defined as the manipulation of materials, processes, or data to misrepresent results and fabrication defined as reporting the results of experiments that were not actually performed (57). Plagiarism refers to the misrepresentation of another's ideas or words as one's own and includes self-plagiarism, sometimes referred to as redundant publication. While some have criticized the term “self-plagiarism” on semantic grounds (7), it has nevertheless proven to be a useful way to describe the practices of publishing the same article in more than one journal or recycling large sections of text in more than one article.

Are retractions becoming more frequent?

Overall, manuscript retraction appears to be occurring more frequently, although it is uncertain whether this is a result of increasing misconduct or simply increasing detection due to enhanced vigilance. Steen reviewed 742 retracted articles and found that the number of retracted articles has risen approximately 10-fold over the past decade, with the greatest increase among those retracted due to misconduct (83). Although errors certainly account for the greatest proportion of retracted articles (56), Steen has argued that many retractions are a consequence of deliberate attempts by an author to deceive (84). Most scientists feel that research misconduct is uncommon. However, a meta-analysis of survey data reported that 2% of scientists report having committed serious research misconduct at least once, and one-third admit to having engaged in questionable research practices (26). Given the stigma associated with retractions and the challenges in detecting misconduct, it is likely that retractions represent only the tip of the iceberg (65). Last year, the journalists Ivan Oransky and Adam Marcus launched a blog called “Retraction Watch,” which is devoted to the examination of retracted articles “as a window into the scientific process” (60); sadly, they seem to have no trouble finding material.

ASM ethical guidelines and retraction policy.

A 2004 survey found that many scientific journals lack formal retraction policies (5). However, the journals of the American Society for Microbiology have specific guidelines for ethical conduct and retractions, which are detailed in the Instructions to Authors (3). These guidelines define plagiarism as well as the fabrication, manipulation, or falsification of data. In addition, the ASM guidelines distinguish between retractions, which are reserved for major errors or misconduct that call the conclusions of an article into question, and errata or authors' corrections, which rectify minor errors. The issue of manipulation of computer-generated images is specifically addressed, with image processing acceptable only if applied to all parts of an image. The interested reader is referred to an excellent commentary by the editors of the Journal of Cell Biology for an extensive discussion of inappropriate digital-image manipulation (68).

Although journals have an important role to play, they do not have primary responsibility for investigating possible scientific misconduct. That responsibility rests with the author's institution (79, 82) and, if funding from the U.S. Department of Health and Human Services is involved, the Office of Research Integrity. Nevertheless, if an editor has concerns about the validity of data in a submitted manuscript, the editor has the prerogative to request that authors provide their raw data for review. If misconduct is suspected, the journal should contact the institution and recommend an inquiry. Once an institution has determined that misconduct involving research publications has occurred, journals are obligated to consider retraction of the work. In the case involving repeated instances of digital-figure manipulation that resulted in six retracted Infection and Immunity articles earlier this year, another journal initially raised the question of misconduct, and the author's institution performed a thorough investigation before informing Infection and Immunity of its concerns. After receiving this notification, Infection and Immunity performed an independent review of the evidence, requested a response from the author(s), and then reached a decision to retract the articles in question after consultation with multiple editors and members of the ASM Publications Board.

Either publishers or authors may initiate a retraction (50, 93). Retraction notices are posted in PubMed and available free of charge, and the pdf versions of retracted articles now carry a watermark to inform readers that the article has been retracted. Authors are consulted regarding the wording of a retraction, but final decisions are at the discretion of the journal. Some journals appear to give authors considerable latitude in wording a retraction notice (23), but this is probably inadvisable (81). The bloggers at Retraction Watch have advocated transparency in retraction notices (59). We concur with the COPE guidelines that notices should state who is issuing the retraction and the reason for the retraction in order to distinguish misconduct from error. The goal in writing a retraction notice is to be clear, accurate, and fair, with fairness applying to both the authors and journal readership. However, beyond this basic information, we are reminded of William Galston's observation that some things must be shrouded “for the same reason that middle-aged people should be clothed” (10).

As a reader once commented to us, “there is no statute of limitation on retractions.” In 1955, Homer Jacobson published an article called “Information, Reproduction and the Origin of Life” in the journal American Scientist (41). Fifty-two years later, after learning that creationists were citing his article as evidence for the divine origin of life, he decided to retract the article (20). Similarly, in 1920 the New York Times published an editorial mocking the aerospace pioneer Robert Goddard for suggesting that a rocket could function in the vacuum of space, stating that Goddard “seems to lack the knowledge ladled out daily in high schools.” The newspaper later retracted their article on 17 July 1969, following the successful launch of Apollo 11 (58).

Can retracted articles be republished?

In theory, a retracted article may be revised and republished, with removal of any erroneous, falsified, fabricated, or plagiarized content. In practice, however, authors of a retracted article may find republication to be a challenge. If misconduct has taken place, the authors may be subject to sanctions from the journal, which prohibit resubmission within a specified time frame. Misconduct compromises the trust between author and editor, and in such cases, authors may find it awkward to later approach the same journal to request consideration of a previously retracted article. In addition, the passage of time may have reduced the significance of the reported findings such that the article is no longer assigned high priority by the journal. Nevertheless, there are instances in which a retracted article has been corrected and republished by the same or another journal (17, 30, 43, 44, 46, 47). Scientists, it would seem, also believe in redemption.

Journals differ in retraction frequency.

To determine whether journals differ in frequency of retracted articles and whether there is a relationship between retraction frequency and journal impact factor, we carried out a PubMed search for retracted articles among 17 journals ranging in impact factor between 2.00 to 53.484. We defined a “retraction index” for each journal as the number of retractions in the time interval from 2001 to 2010, multiplied by 1,000, and divided by the number of published articles with abstracts. A plot of the journal retraction index versus the impact factor revealed a surprisingly robust correlation between the journal retraction index and its impact factor (P < 0.0001 by Spearman rank correlation) (Fig. 1). Although correlation does not imply causality, this preliminary investigation suggests that the probability that an article published in a higher-impact journal will be retracted is higher than that for an article published in a lower-impact journal.

Fig. 1.

Correlation between impact factor and retraction index. The 2010 journal impact factor (37) is plotted against the retraction index as a measure of the frequency of retracted articles from 2001 to 2010 (see text for details). Journals analyzed were Cell, EMBO Journal, FEMS Microbiology Letters, Infection and Immunity, Journal of Bacteriology, Journal of Biological Chemistry, Journal of Experimental Medicine, Journal of Immunology, Journal of Infectious Diseases, Journal of Virology, Lancet, Microbial Pathogenesis, Molecular Microbiology, Nature, New England Journal of Medicine, PNAS, and Science.

The correlation between a journal's retraction index and its impact factor suggests that there may be systemic aspects of the scientific publication process that can affect the likelihood of retraction. When considering various explanations, it is important to note that the economics and sociology of the current scientific enterprise dictate that publication in high-impact journals can confer a disproportionate benefit to authors relative to publication of the same material in a journal with a lower impact factor. For example, publication in journals with high impact factors can be associated with improved job opportunities, grant success, peer recognition, and honorific rewards, despite widespread acknowledgment that impact factor is a flawed measure of scientific quality and importance (8, 29, 33, 77, 80, 86). Hence, one possibility is that fraud and scientific misconduct are higher in papers submitted and accepted to higher-impact journals. In this regard, the disproportionally high payoff associated with publishing in higher-impact journals could encourage risk-taking behavior by authors in study design, data presentation, data analysis, and interpretation that subsequently leads to the retraction of the work. Another possibility is that the desire of high-impact journals for clear and definitive reports may encourage authors to manipulate their data to meet this expectation. In contradistinction to the crisp, orderly results of a typical manuscript in a high-impact journal, the reality of everyday science is often a messy affair littered with nonreproducible experiments, outlier data points, unexplained results, and observations that fail to fit into a neat story. In such situations, desperate authors may be enticed to take short cuts, withhold data from the review process, overinterpret results, manipulate images, and engage in behavior ranging from questionable practices to outright fraud (26). Alternatively, publications in high-impact journals have increased visibility and may accordingly attract greater scrutiny that results in the discovery of problems eventually leading to retraction. It is possible that each of these explanations contributes to the correlation between retraction index and impact factor. Whatever the explanation, the phenomenon appears deserving of further study. The relationship between retraction index and impact factor is yet another reason to be wary of simple bibliometric measures of scientific performance, such as impact factor.

Impact of research misconduct.

Science must try to be self-correcting, and retractions provide a critically important function by rectifying the scientific record. However, the system is far from perfect. As we have already noted, it is likely that only a small percentage of scientific misconduct results in retraction. Sensational new claims attract scrutiny and are more likely to be refuted by subsequent research (2, 34, 35, 64, 69–76, 95). However, reports based on falsified or fabricated data may be more difficult to detect if the conclusions happen to be true. Retractions often do not occur for years after publication (1, 18, 21, 90), which is perhaps understandable given the time required for other researchers to attempt to replicate results and for institutions to perform thorough investigations (100), but this means that erroneous information remains in circulation for prolonged periods before correction (62). Moreover, it is disheartening that retracted articles continue to be cited, sometimes for decades afterward (11, 24, 45, 63, 66, 88, 96).

It is not difficult to surmise the underlying causes of research misconduct. Misconduct represents the dark side of the hypercompetitive environment of contemporary science, with its emphasis on funding, numbers of publications, and impact factor (39). With such potent incentives for cheating, it is not surprising that some scientists succumb to temptation. As Eric Poehlman, an obesity researcher sentenced to jail for research misconduct, said at his sentencing hearing, “I had placed myself…in an academic position in which the amount of grants that you held basically determined one's self worth…everything flowed from that” (36). Funding agencies and journals provide regulations and disincentives for misconduct, but these may be inadequate if the incentives are too great and even counterproductive if the penalties are excessively harsh. Another response to misconduct has been to increase formal ethics instruction for research trainees. While this effort may be worthwhile, there is little evidence of its effectiveness (31). When a prominent article is retracted, a common refrain is, “Why didn't the reviewers catch that?” In fact, many would-be retractions are caught during the review process. However, without access to raw data, it is unrealistic to expect that even careful and highly motivated reviewers can detect all instances of falsification or fabrication.

Plagiarism is a more complex matter, as it is based upon a modern concept of intellectual property that dates back only to 18th-century Europe (12). The rise of the internet has facilitated plagiarism, but technology has also arisen to facilitate the detection of plagiarism or redundant publication (25, 32, 48, 92). Some have suggested that plagiarism is a culturally relative concept, which is less likely to be regarded as an unethical practice by some scientists in non-Western countries or those belonging to the younger generation (9, 22, 40, 91, 97–99). However, we do not share this view. Scientists must be explorers, and it is best if they do not precisely follow the wagon ruts left by their predecessors but instead strike out on their own paths, using their own words. The ASM journals strictly prohibit plagiarism and self-plagiarism.

Conclusions.

The increasing rate of retracted scientific articles is a disturbing trend. Although correction of the scientific record is laudable per se, erroneous or fraudulent research can cause enormous harm, diverting other scientists to unproductive lines of investigation, leading to the unfair distribution of scientific resources, and in the worst cases, even resulting in inappropriate medical treatment of patients (6, 85). Furthermore, retractions can erode public confidence in science. Any retraction represents a tremendous waste of scientific resources that are often supported with public funding, and the retraction of published work can undermine the faith of the public in science and their willingness to provide continued support. The corrosive impact of retracted science is disproportionate to the relatively small number of retracted articles. The scientific process is heavily dependent on trust. To the extent that misconduct erodes scientists' confidence in the literature and in each other, it seriously damages science itself. As Arst has noted, “All honest scientists are victims of scientists who commit misconduct” (4). Yet, retractions also have tremendous value. They signify that science corrects its mistakes.

ADDENDUM IN PROOF

It has recently been brought to our attention that previous independent analyses have also concluded that articles in journals with higher impact factors are more likely to be retracted (Liu, S. V. Top journals' top retraction rates. Sci. Ethics 1:91–93, 2006; Cokol, M., I. lossifov, R. Rodriguez-Esteban, and A. Rzhetsky. How many scientific papers should be retracted? EMBO Rep. 8:422–423, 2007).