Misconduct accounts for the majority of retracted scientific publications

Abstract

A detailed review of all 2,047 biomedical and life-science research articles indexed by PubMed as retracted on May 3, 2012 revealed that only 21.3% of retractions were attributable to error. In contrast, 67.4% of retractions were attributable to misconduct, including fraud or suspected fraud (43.4%), duplicate publication (14.2%), and plagiarism (9.8%). Incomplete, uninformative or misleading retraction announcements have led to a previous underestimation of the role of fraud in the ongoing retraction epidemic. The percentage of scientific articles retracted because of fraud has increased ∼10-fold since 1975. Retractions exhibit distinctive temporal and geographic patterns that may reveal underlying causes.

The number and frequency of retracted publications are important indicators of the health of the scientific enterprise, because retracted articles represent unequivocal evidence of project failure, irrespective of the cause. Hence, retractions are worthy of rigorous and systematic study. The retraction of flawed publications corrects the scientific literature and also provides insights into the scientific process. However, the rising frequency of retractions has recently elicited concern (1, 2). Studies of selected retracted articles have suggested that error is more common than fraud as a cause of retraction (3–5) and that rates of retraction correlate with journal-impact factor (6). We undertook a comprehensive analysis of all retracted articles indexed by PubMed to ascertain the validity of the earlier findings. Retracted articles were classified according to whether the cause of retraction was documented fraud (data falsification or fabrication), suspected fraud, plagiarism, duplicate publication, error, unknown, or other reasons (e.g., journal error, authorship dispute).

Results

Causes of Retraction.

PubMed references more than 25 million articles relating primarily to biomedical research published since the 1940s. A comprehensive search of the PubMed database in May 2012 identified 2,047 retracted articles, with the earliest retracted article published in 1973 and retracted in 1977. Hence, retraction is a relatively recent development in the biomedical scientific literature, although retractable offenses are not necessarily new. To understand the reasons for retraction, we consulted reports from the Office of Research Integrity and other published resources (7, 8), in addition to the retraction announcements in scientific journals. Use of these additional sources of information resulted in the reclassification of 118 of 742 (15.9%) retractions in an earlier study (4) from error to fraud. A list of 158 articles for which the cause of retraction was reclassified because of consultation of secondary sources is provided in Table S1. For example, a retraction announcement in Biochemical and Biophysical Research Communications reported that “results were derived from experiments that were found to have flaws in methodological execution and data analysis,” giving the impression of error (9). However, an investigation of this article conducted by Harvard University and reported to the Office of Research Integrity indicated that “many instances of data fabrication and falsification were found” (10). In another example, a retraction notice published by the authors of a manuscript in the Journal of Cell Biology stated that “In follow-up experiments . . . we have shown that the lack of FOXO1a expression reported in figure 1 is not correct” (11). A subsequent report from the Office of Research Integrity states that the first author committed “research misconduct by knowingly and intentionally falsely reporting . . . that FOXO1a was not expressed . . . by selecting a specific FOXO1a immunoblot to show the desired result” (12). In contrast to earlier studies, we found that the majority of retracted articles were retracted because of some form of misconduct, with only 21.3% retracted because of error. The most common reason for retraction was fraud or suspected fraud (43.4%), with additional articles retracted because of duplicate publication (14.2%) or plagiarism (9.8%). Miscellaneous reasons or unknown causes accounted for the remainder. Thus, for articles in which the reason for retraction is known, three-quarters were retracted because of misconduct or suspected misconduct, and only one-quarter was retracted for error.

Fig. 1.

(A) Number of retracted articles for specific causes by year of retraction. (B) Percentage of published articles retracted for fraud or suspected fraud by year of publication.

Temporal Trends.

A marked recent rise in the frequency of retraction was confirmed (2, 13), but was not uniform among the various causes of retraction (Fig. 1A). A discernible rise in retractions because of fraud or error was first evident in the 1990s, with a subsequent dramatic rise in retractions attributable to fraud occurring during the last decade. A more modest increase in retractions because of error was observed, and increasing retractions because of plagiarism and duplicate publication are a recent phenomenon, seen only since 2005. The recent increase in retractions for fraud cannot be attributed solely to an increase in the number of research publications: retractions for fraud or suspected fraud as a percentage of total articles have increased nearly 10-fold since 1975 (Fig. 1B).

Geographic Origin and Impact Factor.

Retracted articles were authored in 56 countries, and geographic origin was found to vary according to the cause for retraction (Fig. 2). The United States, Germany, Japan, and China accounted for three-quarters of retractions because of fraud or suspected fraud. China and India collectively accounted for more cases of plagiarism than the United States, and duplicate publication exhibited a pattern similar to that of plagiarism. The relationship between journal impact factor and retraction rate was also found to vary with the cause of retraction. Journal-impact factor showed a highly significant correlation with retractions because of fraud or error but not with those because of plagiarism or duplicate publication (Fig. 3 A–C). Moreover, the mean impact factors of journals retracting articles because of fraud or error differed significantly from that of journals retracting articles because of plagiarism or duplicate publication. Accordingly, retractions for fraud or error and retractions for plagiarism or duplicate publication were encountered in distinct subsets of journals, with differences in impact factor (Fig. 3D) and limited overlap (Table 1).

Fig. 2.

Country of origin of publications retracted for fraud or suspected fraud (A), plagiarism (B), or duplicate publication (C).

Fig. 3.

Relation of journal-impact factor to retractions for fraud or suspected fraud, error, and plagiarism, or duplicate publication. Journal-impact factor showed a highly significant correlation with the number of retractions for fraud or suspected fraud (A) (n = 889 articles in 324 journals, R² = 0.08664, P < 0.0001) and error (B) (n = 437 articles in 218 journals, R² = 0.1142, P < 0.0001), and a slight correlation with the number of retractions for plagiarism or duplicate publication (C) (n = 490 articles in 357 journals, R² = 0.01420, P = 0.0243). The mean journal-impact factor of articles retracted because of fraud/suspected fraud or error was significantly different from that of papers retracted because of plagiarism or duplicate publication (D) (error bars ± SEM, P < 0.0001).

Table 1.

Journals with most retracted articles

Journal	No. of articles	IF
Total
Science	70	32.45
Proceedings of the National Academy of Sciences	69	10.47
The Journal of Biological Chemistry	54	5.12
Nature	44	36.24
Anesthesia & Analgesia	40	3.07
The Journal of Immunology	34	5.86
Blood	28	9.79
The Journal of Clinical Investigation	23	15.43
Cell	22	34.77
Biochemical and Biophysical Research Communications	18	2.52
The New England Journal of Medicine	16	50.08
The EMBO Journal	15	8.83
Journal of Hazardous Materials	15	4.55
Molecular and Cellular Biology	15	5.77
Infection and Immunity	14	4.06
Fraud/suspected fraud
The Journal of Biological Chemistry	37	5.12
Anesthesia & Analgesia	33	3.07
Science	32	32.45
The Journal of Immunology	30	5.86
Proceedings of the National Academy of Sciences	27	10.47
Blood	21	9.79
Nature	19	36.24
The Journal of Clinical Investigation	17	15.43
Cancer Research	16	8.16
Cell	13	34.77
Journal of Hazardous Materials	13	4.55
British Journal of Anaesthesia	11	3.85
The EMBO Journal	11	8.83
The New England Journal of Medicine	11	50.08
International Journal of Cancer	10	4.92
Molecular and Cellular Biology	10	5.77
Error
Science	37	32.45
Proceedings of the National Academy of Sciences	36	10.47
Nature	25	36.24
Biochemical and Biophysical Research Communications	7	2.52
Cell	7	34.77
The Journal of Biological Chemistry	7	5.12
Journal of Virology	7	5.32
The Lancet	6	33.80
Anesthesiology	5	5.19
Infection and Immunity	5	4.06
11 journals*	4*	–
Plagiarism/duplicate publication
Molecules and Cells	8	1.99
Phytotherapy Research	7	2.47
Biotechnology Advances	5	10.96
Aesthetic Plastic Surgery	5	1.50
Annals of the New York Academy of Sciences	5	3.00
Journal of Child and Adolescent Psychiatric Nursing	4	None
Archives of Iranian Medicine	4	0.97
Resuscitation	4	3.02
Clinical Rheumatology	4	1.72
The New England Journal of Medicine	4	50.08
International Journal of Cardiology	4	4.11
Anesthesia & Analgesia	4	3.07
Biochemical and Biophysical Research Communications	4	2.52
16 journals†	3†	–

IF, impact factor.

*Biochemistry, Blood, Contraception, The EMBO Journal, Journal of the American Chemical Society, The Journal of Clinical Investigation, Journal of Ethnopharmacology, The Journal of Immunology, Molecular and Cellular Biology, Neurology, and Plant Physiology.

^†Blood, Cardiovascular Research, Croatian Medical Journal, Current Eye Research, European Journal of Medicinal Chemistry, FEMS Immunology & Medical Microbiology, Journal of Cellular Physiology, Journal of Clinical Oncology, Journal of Experimental & Clinical Cancer Research, Journal of Medical Primatology, Journal of the Pakistan Medical Association, Langmuir, Neurosurgery, Transplant Proceedings, Viral Immunology, and Yonsei Medical Journal.

Time-to-Retraction.

The time interval between publication and retraction varied according to the cause of retraction, with articles retracted because of fraud taking substantially longer to retract (Table 2). A gradual trend toward increasing time-to-retraction over time was detected (Fig. 4A). Journal-impact factor did not correlate with time-to-retraction for manuscripts retracted because of error, plagiarism, or duplicate publication, but did exhibit a modest correlation for manuscripts retracted because of fraud in high-impact journals, which tended to exhibit a shorter time-to-retraction (Fig. 4B). A small number of authors were responsible for multiple retractions. Thirty-eight research groups with greater or equal to five retractions accounted for 43.9% (n = 390) of retractions for fraud or suspected fraud in the modern biomedical literature (Fig. S1). Nearly all retracted articles by authors with 10 or more retractions were retracted because of fraud (Table S2). This finding is attributable to the discovery of multiple fraudulent articles during the course of investigation of a single instance of fraud. For example, the retraction of a 2010 Blood article by Sawada et al. (14) was followed in rapid succession by the retraction of 30 additional articles originating from the laboratory of Naoki Mori (Fig. S2).

Table 2.

Mean time-to-retraction by category

Cause of retraction	n	Months to retract (Mean)	SD
All causes*	2,047	32.9	34.2
Fraud (fabrication/falsification)	697	46.8	38.4
Suspected fraud	192	29.4	30.0
Plagiarism	200	26.0	32.6
Duplicate publication	290	27.0	30.1
Error	437	26.0	28.0
Other	108	19.8	31.1
Unknown	182	22.1	25.4

*Some articles fall into more than one category.

Fig. 4.

(A) Time-to-retraction as a function of year of retraction. R² = 0.1236, P = 0.0414. (B) Time-to-retraction as a function of impact factor. Journal-impact factor correlated inversely with time-to-retraction for articles retracted because of fraud (n = 697, R² = 0.01441, P = 0.0015) but not other causes.

Citation of Retracted Articles.

Previous investigators have found that many retracted articles continue to be cited as if still valid work (15, 16), but others have documented an immediate effect of retraction on citation frequency (17). Although we did not examine this question comprehensively, we found considerable variation among the most frequently cited retracted articles (Table 3). Some retracted articles exhibited a rapid and sustained decline in citations following retraction, but others have continued to be cited (Fig. S3).

Table 3.

Most cited retracted articles

First author	Journal	Year published	Year retracted	Times cited*	Reason for retraction
Wakefield	Lancet	1998	2004; 2010	758	Fraud
Reyes	Blood	2001	2009	740	Error
Fukuhara	Science	2005	2007	686	Error
Nakao	Lancet	2003	2009	626	Fraud
Chang	Science	2001	2006	512	Error
Kugler	Nature Medicine	2000	2003	494	Fraud
Rubio	Cancer Research	2005	2010	457	Error
Gowen	Science	1998	2003	395	Fraud
Makarova	Nature	2001	2006	375	Error
Hwang	Science	2004	2006	368	Fraud
Potti	The New England Journal of Medicine	2006	2011	361	Fraud
Brugger	The New England Journal of Medicine	1995	2001	336	Fraud
Van Parijs	Immunity	1999	2009	330	Fraud
Potti	Nature Medicine	2006	2011	328	Fraud
Schön	Science	2000	2002	297	Fraud
Chiu	Nature	2005	2010	281	Error
Cooper	Science	1997	2005	264	Fraud
Le Page	Cell	2000	2005	262	Error
Kawasaki	Nature	2004	2006	243	Fraud
Hwang	Science	2005	2006	234	Error

*As of June 22, 2012.

Discussion

In addition to confirming a recent rise in the incidence of retractions, this study provides a number of additional insights. Perhaps most significantly, we find that most retracted articles result from misconduct, and nearly half of retractions are for fraud or suspected fraud. In addition to a larger sample size encompassing all retractions in the biomedical research literature, this study differs from some previous analyses in the use of alternative sources of information, such as reports from the Office of Research Integrity, Retraction Watch, news media, and other public records. The US Office of Research Integrity was formed in 1992 and is charged with the oversight of misconduct allegations involving federally sponsored research. As the consideration of secondary sources led to changes in the perceived cause of retraction in 158 instances (Table S1), we conclude that for many retractions, the retraction notice is insufficient to ascertain the true cause of a retraction.

We further note that not all articles suspected of fraud have been retracted. The Lancet and British Medical Journal expressed serious reservations about the validity of the Indo-Mediterranean Diet Heart Study after the primary author was unable to present original records to document ethics review and informed consent (18, 19), yet the original articles have not been retracted (20, 21). Several articles authored by Mark Spector when he was working in the laboratory of Efraim Racker remain in the literature (22, 23), despite documentation that Spector committed data fabrication (24). R. K. Chandra was found to have committed fraud in the performance of clinical trials, but only a single article was retracted (25), even though considerable evidence was obtained to suggest that other publications were also fraudulent (26). Therefore, the current number of articles retracted because of fraud represents an underestimation of the actual number of fraudulent articles in the literature.

Although some retraction announcements are specific and detailed, many are uninformative or opaque. In 119 instances, no information regarding the reason for retraction was provided by the journal. Announcements are often written by the authors of the retracted article themselves (27), who may be understandably reluctant to implicate themselves in misconduct. Furthermore, investigation of suspected misconduct is a lengthy process, and retraction notices are frequently made before the full results of investigations are available. Among 285 investigations concluded by the Office of Research Integrity from 2001 to 2010, the length of investigation averaged 20.48 mo in duration and ranged up to more than 9 y (28). Policies regarding retraction announcements vary widely among journals, and some, such as the Journal of Biological Chemistry, routinely decline to provide any explanation for retraction. These factors have contributed to the systematic underestimation of the role of misconduct and the overestimation of the role of error in retractions (3, 4), and speak to the need for uniform standards regarding retraction notices (5).

Differences in the temporal and geographic patterns of retraction according to cause (Figs. 1A and 2) militate against a simple explanation for retractions. One factor is the increased detection of misconduct. The first discernible increase in retractions followed the formation of the Office of Scientific Integrity (the predecessor of the Office of Research Integrity) and passage of the Whistleblower Protection Act in 1989. The recent increase in the incidence of retractions and the differing patterns by region (Fig. 2) argue that incentives may vary with the type of misconduct. Most articles retracted for fraud have originated in countries with longstanding research traditions (e.g., United States, Germany, Japan) and are particularly problematic for high-impact journals. In contrast, plagiarism and duplicate publication often arise from countries that lack a longstanding research tradition, and such infractions often are associated with lower-impact journals (Fig. 3 and Table 1). A highly significant correlation was found between the journal-impact factor and the number of retractions for fraud or suspected fraud and error (Fig. 3 A and B); the mean impact factor was found to be significantly higher for articles retracted for fraud, suspected fraud, or error, compared with those retracted for plagiarism or duplicate publication (Fig. 3D). An association between impact factor and retraction for fraud or error has been noted previously (4, 6, 29, 30). This finding may reflect the greater scrutiny accorded to articles in high-impact journals and the greater uncertainty associated with cutting-edge research. Alternatively, the disproportionately high payoffs to scientists for publication in prestigious venues can be an incentive to perform work with excessive haste (31) or to engage in unethical practices (4). The modest correlation between impact factor and time-to-retraction argues against an explanation based on increased scrutiny alone, but the higher proportion of fraud in highly prestigious journals is consistent with the suggestion that the benefits of publishing in such venues are powerful incentives for fraud (4, 6, 32). The 20 most highly cited retracted articles (Table 3) include no articles retracted for plagiarism or duplicate publication. Recent technological advances facilitating word processing and access to electronic publications may have made plagiarism easier to perform but have also facilitated its detection (33). The impact of plagiarism and duplicate publication on science is therefore limited; although these practices are to be discouraged, the present study suggests that the reduction of fraud and error should be a higher priority (32).

Our findings confirm that retraction can cause a persistent decline in citation rate (17), but an immediate and severe decline in citations is not inevitable (Fig. S3). The Wakefield article in the Lancet (34) is a special case, given its sequential partial and full retractions, extensive media attention, public health import, and frequent citation as a source of controversy, but it is less obvious why other retracted articles continue to be cited. A PDF version of a 2001 Nature article by Makarova et al. does not indicate that the article has been retracted, and two authors disputed whether iron-containing impurities invalidated the articles conclusions (35), which might contribute to uncertainty as to whether the work remains valid (36). A 2005 Science article by Fukuhara et al. continues to be cited even though both the HTML and PDF versions are clearly marked as retracted, and the PDF version includes a copy of the retraction notice (37). Many scientists continue to cite the article by Fukuhara et al. for its initial identification of visfatin as an adipocytokine, even though the article was retracted because of concerns about the cytokine’s reported insulin mimetic properties (38). This practice suggests that under certain circumstances, scientists continue to find utility in retracted articles, particularly those retracted because of error, and supports a policy of continued access to retracted articles as long as detailed descriptions regarding the reasons for retraction are provided.

The longer average time-to-retraction when articles are retracted because of fraud (Table 2) corroborates earlier studies based on limited datasets (4, 39), and may be attributable in part to the lengthy investigative process required to establish misconduct. The trend toward a longer average time-to-retraction for articles retracted in recent years may reflect a growing tendency for editors to reach back further in time to retract articles. Recognition of fraud can trigger a systematic review of an author’s entire research output, resulting in a cascade of retracted articles. The correlation between time-to-retraction and impact factor (Fig. 4B) suggests that the greater visibility and enhanced scrutiny of high-impact journals may contribute to more rapid retraction of fraudulent papers by these journals, although the effect appears to be quite modest.

Most articles by authors with large numbers of retractions (Table S2) were retracted because of misconduct, and these include some of the most notorious cases in the history of research ethics. The Mori case (Fig. S2) demonstrates that fraudulent articles can go undetected for many years. Such cases may be revealed only fortuitously when exposed by an attentive reviewer or whistleblower (40). Twelve of Mori’s retracted articles had been in the literature for 5 y or more, demonstrating that the impact of serial retractions on the average time-to-retraction can be substantial.

In conclusion, a comprehensive review of 2,047 articles retracted from the biomedical literature reveals that misconduct has played a more prominent role than previously appreciated. Our findings underscore the importance of vigilance by reviewers, editors, and readers, and investigations by institutions, government agencies, and journalists in identifying and documenting research misconduct. Furthermore, our findings suggest a need for increased attention to ethics in the training of scientists. However, this attention alone is unlikely to be successful in curbing poor research practices.

The rise in the rate of retractions raises concern about the health of the scientific enterprise itself (32). Although articles retracted because of fraud represent a very small percentage of the scientific literature (Fig. 1B), it is important to recognize that: (i) only a fraction of fraudulent articles are retracted; (ii) there are other more common sources of unreliability in the literature (41–44); (iii) misconduct risks damaging the credibility of science; and (iv) fraud may be a sign of underlying counterproductive incentives that influence scientists (45, 46). A better understanding of retracted publications can inform efforts to reduce misconduct and error in science.

Given that most scientific work is publicly funded and that retractions because of misconduct undermine science and its impact on society, the surge of retractions suggests a need to reevaluate the incentives driving this phenomenon. We have previously argued that increased retractions and ethical breaches may result, at least in part, from the incentive system of science, which is based on a winner-takes-all economics that confers disproportionate rewards to winners in the form of grants, jobs, and prizes at a time of research funding scarcity (32, 46, 47). We have also proposed a set of reforms to strengthen the scientific enterprise, ranging from improved training of scientists to the identification of mechanisms to provide more consistent funding for science (32, 46). Solutions to address the specific problem of retractions may include the increased use of checklists by authors and reviewers, improved training in logic, probability and statistics, an enhanced focus on ethics, the formation of a centralized database of scientific misconduct, the establishment of uniform guidelines for retractions and retraction notices, and the development of novel reward systems for science (32). Dedicated national agencies, such as the US Office for Research Integrity, can play an invaluable role in supporting and overseeing institutional investigations of alleged misconduct. We hope that the present study will prompt discussion among scientists and the society they serve to find measures to improve the quality and reliability of the scientific literature.

Methods

The database used for this study was compiled from a search of all articles indexed by PubMed as retracted publications in English on May 3, 2012. Articles were classified according to cause of retraction as fraud, suspected fraud, error, plagiarism, duplicate publication, other, or unknown on the basis of retraction announcements. In addition, retracted articles were cross-checked against the annual reports of the Office of Research Integrity. An internet search using the Google search engine was performed to seek additional information regarding retracted articles for which the reason for retraction remained unclear, and included Retraction Watch, news media, and other public records. In one case, an author was contacted to clarify a published retraction notice. Each classification decision was independently reviewed by all authors and any discrepancies were resolved. Impact factors were based on the 2011 edition of Journal Citation Reports Science Edition (Thomson Reuters, released June 28, 2012) (48), and 5-y impact factors were used when available. In one case (Acta Crystallographica A), the 2011 impact factor (2.076) was used instead of the 5-y impact factor (30.646), because the latter was felt to be anomalous. Journals without an impact factor were assigned a value of 0.1. Statistical analyses were performed using Prism (GraphPad Software).