RESEARCH MISCONDUCT POLICIES OF SCIENTIFIC JOURNALS

Abstract

The purpose of this study was to gather information on the misconduct policies of scientific journals. We contacted editors from a random sample of 399 journals drawn from the ISI Web of Knowledge database. We received 197 responses (49.4% response rate): 54.8% had a policy, and 47.7% had a formal (written) policy; 28.9% had a policy that only outlined procedures for handling misconduct, 15.7% had a policy that only defined misconduct, 10.2% had a policy that included both a definition and procedures; 26.9% of journals had a policy that was generated by the publisher, 13.2% had a policy that was generated by the journal, and 14.7% had a policy that was generated by another source, such as a professional association. We analyzed the relationship between having a policy and impact factor, field of science, publishing house, and nationality. Impact factor was the only variable with a statistically significant association with having a policy. Impact factor was slightly positively associated with whether or not the publisher had a policy, with an odds ratio of 1.49 (P < .0004) per 10 units increase in the impact factor, with a 95% confidence interval (1.20, 1.88). Our research indicates that more than half of scientific journals have developed misconduct policies, but that most of these policies do not define research misconduct and most of these policies were not generated by the journal.

Introduction

Dealing with research misconduct in published articles can be complex and onerous, as well as ethically challenging and legally risky (Couzin and Unger, 2006). Editors must decide how to define misconduct, handle allegations of misconduct, and correct the publication record when presented with confirmed cases of misconduct. In the wake of recent highly-publicized scandals, such as data fabrication and falsification committed by South Korean stem cell scientist Woo Suk Hwang and U.S. medical researcher Eric Poehlmam, some journals have begun to rethink their editorial policies (Cyranoski, 2006; Kinitisch, 2005; Resnik, 2008). The Committee on Publication Ethics, which was formed in 1997, developed some guidelines for responding to misconduct in research in 2004 (Committee on Publication Ethics, 2004). The Council of Science editors published a white paper in 2006 on integrity in scientific publication that defined misconduct and addressed how journals should deal with misconduct (Council of Science Editors, 2006).

Although journals have been addressing misconduct issues, there is little published research about journal misconduct policies. Redman and Merz (2006) published a study of the misconduct policies of 50 biomedical journals with the highest impact. The authors located misconduct policies on journal Web sites and contacted editors by email and phone concerning those policies. The authors showed that only 14% of the journals in their sample had developed misconduct policies. However, their study had some significant limitations, since the sample was small and not random, and they had a low response rate (18%) from editors, which could contribute to substantial underreporting. A recent study of journal conflict of interest policies found that many journals had policies they had not posted on their websites (Ancker and Flanagin, 2007). The purpose of our study was to conduct a more thorough and systematic study of the misconduct policies of scientific journals.

Materials and Methods

We selected the journals for our study from ISI Web of Knowledge, Journal Citation Reports, one of the premier electronic databases for scientific research. The Journal Citation Reports database contained 5,838 journals from the physical sciences, engineering, the biomedical sciences, and the social sciences when we did our search on June 15, 2008 (ISI Web of Knowledge, 2008). We sorted this sample according to impact factor, generated a random sequence of numbers using the Random.org website, and selected 400 journals from this random sequence (Random.org, 2008). Since one of these journals was a duplicate, we actually contacted only 399 journals. We choose to contact 400 because we wanted to achieve a 5% margin of error for our main variable, the number of journals with misconduct policies. To achieve a 5% margin of error, we would need a sample size of 138, since we estimated that only 10% of journals would have policies. We also estimated that our response rate would be only 40%. So, to get 138 responses, we would need to contact 345 journals. We decided to contact 400 to ensure that we would get at least 138 responses.

To contact the journals, we searched for their websites using internet search engines, looking for editors or editorial staff to contact. If an e-mail address was provided, we used it. When we contacted the journals, we sent them an email explaining our study and asked them to provide us with information about their misconduct policies, if they had any. We asked them to send us a copy of their policy or a link to a website. If we did not receive a response within 10 days, we sent out a second reminder e-mail to the editors. A list of respondents is provided in the Appendix.

We defined a misconduct policy as rules or statements about the definition of misconduct or procedures for responding to misconduct, such as how to report allegations of misconduct or correcting the scientific literature in response to confirmed cases of misconduct. The U.S. federal government defines research misconduct as fabrication, falsification, or plagiarism (FFP) (Office of Science and Technology Policy, 2005). We did not limit definitions of misconduct to only FFP, because many research organizations recognize types of misconduct other than FFP, such as misuse of human or animal subjects in research (Resnik, 2008). We did not consider journal policies pertaining to duplicate publication, simultaneous submission, or copyright permissions to be misconduct policies.

We categorized responses as follows: a) misconduct policy (yes or no); type of policy (formal, i.e., written, informal, or no policy); source of policy (journal-generated, publisher-generated, other, or no policy); and content of policy (definition of misconduct, procedures for responding to misconduct, both, no policy). We also collected information about the journals using the ISI website. This information included: impact factor, publisher, field of science (physical sciences and engineering, biomedical sciences, and social sciences), and country of origin of the journal.

We used logistic regression analysis in the statistical software package SAS, version 9.1 to analyze the data. We regressed the log odds that a journal has a misconduct policy against several variables, including publisher, country of origin of the journal, field of science, and impact factor. The NIH’s Office of Human Research Protections classified our study as exempt research.

Results

One Hundred ninety seven out of 399 journals responded to our request for information (49.4% response rate): 54.8% of journals had a misconduct policy, while 45.2% had no policy; 47.7% of journals a formal policy, and 7.1% had an informal policy; 28.9% of journals had a policy that addressed procedures for dealing with misconduct but did not include a definition of misconduct, 15.7% had a policy that defined misconduct but did not specify procedures for dealing with misconduct, and 10.2% had a policy that included procedures and a definition; 26.9% of journals had a policy that was generated by the publisher, 13.2% had generated their own policy, and 14.7% had a policy that was generated by some other source, such as a professional society. The impact factor for the journals in our sample ranged from 0 to 33.5, with a mean of 2.23 and a standard deviation of 3.05; 48.7% of journals were based in Europe or Australia, 41.1% were based in the U.S. or Canada, and 10.2% were based somewhere else, such as China or Russia; 61.4% of the journals were from the biological sciences, 33% were from the physical sciences or engineering, and 5.6% were from the social sciences. The results are summarized in Table 1.

TABLE 1.

Summary of Data on Misconduct Poliies (n =197 for All Percentages)

Journal Has a Misconduct Policy _____
Yes	54.8% (S.E. 3.5)
No	45.2% (S.E. 3.5)
Type of Policy _____
Formal	47.7% (S.E. 3.5)
Informal	7.1% (S.E. 1.8)
No policy	45.2% (S.E. 3.5)
Content of Policy _____
Procedures	28.9% (S.E. 3.2)
Definition	15.7% (S.E. 2.6)
Procedures and definition	10.2% (S.E. 2.1)
No policy	45.2% (S.E. 3.5)
Source of Policy _____
Publisher	26.9% (S.E. 2.5)
Journal	13.2% (S.E. 2.4)
Other	14.7% (S.E. 2.5)
No policy	45.2% (S.E. 3.5)
Impact Factor of Journals in the Sample _____
Range: 0–33.5; Mean: 2.23; S.D.: 3.05
Country of Origin of Journals _____
Western Europe and Australia	48.7%
U.S. and Canada	41.1%
Other	10.2%
Publisher of Journals _____
Elsevier	23.4%
Wiley	10.7%
Springer	7.6%
Other	58.3%
Field of Science _____
Biological sciences	61.4%
Physical sciences and engineering	33%
Social sciences	5.6%

In our data analysis, we combined some of the variables due to low numbers or the need to achieve a more even distribution among categories. Due to the low number of social science journals in our sample, we combined the social sciences with the physical sciences and engineering. We combined Western European countries and Australia, and the U.S. and Canada. All other countries were placed in the “other” category. To get an even distribution, publishers were divided into 2 groups: the most common ones (Elsevier, Wiley, and Springer) and others.

We performed logistic regression analysis to determine the association between having a misconduct policy and (i) the publishing house (Elsevier/Wiley/Springer or Other), (ii) field of science (biological sciences or physical/social sciences/engineering), (iii) country of publication, and (iv) impact factor. We used PROC GENMOD in SAS 9.0. We found that the impact factor was slightly positively associated with whether or not the publisher had a misconduct policy, with an odds ratio of 1.49 (P < .0004) per 10 units increase in the impact factor, with a 95% confidence interval (1.20, 1.88). The remaining variables are not significant at P = .05. The results are summarized in the Table 2.

TABLE 2.

Association Between Having a Misconduct Policy and Four Factors

Factor	P-value
Country of journal	.08
Field of science	.53
Publisher of journal	.38
Impact factor of journal	.0004

Discussion

The most significant finding of our study is that more than half of the journals in the sample had a misconduct policy and that nearly half had a formal (written) policy. This is a much larger percentage than reported by Redman and Merz (2006). It is difficult to say whether the difference between our results and those of Redman and Merz is due to changes among journal policies since they did their study or as the result of differences in our methodologies. We suspect that the latter is the case, due to the small sample size of the Redman and Merz study and their low response rate, but this question requires further investigation.

Another significant finding of our study is that publishing houses play an important role in providing misconduct policies for journals. More than half of the journals that had misconduct policies had policies that were provided by the publisher; 23.4% of the journals in our study were published by Elsevier, and many Elsevier journals are now members of COPE, which has extensive misconduct policies (Committee on Publication Ethics, 2009). Elsevier’s decision to request that its journals become members of COPE has, therefore, had a significant impact on the adoption of misconduct policies by scientific journals.

It is also important to note that only 10.2% of the journals had a policy that included procedures for dealing with misconduct and a definition of misconduct. Both of these types of policies are important, since procedures are necessary for protecting human rights and ensuring due process and fairness, and a definition of misconduct is important for determining when a journal should take action in response to an allegation or finding of misconduct. Though many journals have made considerable strides toward developing misconduct policies, more work is needed. The need for additional work on this problem is especially important, given the global nature of scientific research and a lack of consensus among different countries concerning misconduct policies (Resnik, 2003; Momen and Gollogly, 2007).

Another important finding of our study is that having a policy was positively associated with impact factor. Although impact factor had only a small effect, the effect was highly statistically significant (P = .0004). Some potential explanations for this association are that high impact factor journals have had more incidents of misconduct (so there is a greater need to develop policies), high impact journals receive more scrutiny and publicity than lower impact journals (and hence, they need to develop policies in response to extra attention), and the stakes (e.g., significant discoveries or financial gains) are often higher in high impact journals than in lower ones. These possible explanations are only speculative at this point, and the issues require further study.

One of the limitations of our study is that our response rate is slightly below 50%. A possible explanation of this response rate is that we sent out our e-mail inquiries during June and July 2008, and many journal editors may have been on summer vacation or out of the office at that time. Additionally, some of our e-mails may have been screened out by spam filtering programs, and some of the contact information we used for editors may have been outdated. Some journals may not have responded to our inquiry because they expected that we could find the information through the journal’s or publisher’s website. We do not believe that any of these factors that may have affected the response rate have also biased our results.

Another limitation of the study is that our sample was not large enough to determine the impact of social science as a variable. As noted earlier, since we had only 11 journals from the social sciences, we had to combine this variable with the physical sciences for the purposes of our data analysis. Future studies conducted by other investigators could focus more directly on social science journals.

It is worth noting that journal impact factors change frequently. We collected data on journal impact factors for the time frame in which we conducted our study, not earlier. If a journal had a misconduct policy, the impact factor that we recorded for the journal could be different from the impact factor at the time the journal adopted the policy. We do not believe that this potential difference in impact factors significantly affected our results, and that impact factor is still a good predictor of whether a journal has a misconduct policy.

Finally, it is worth noting that we did not specifically ask journals whether they had had a misconduct incident in recent years. It would have been interesting to ascertain whether there is a relationship between having a misconduct incident and developing a policy. Many respondents provided this information, even though we did not request it. 12.2% of respondents reported that they had dealt with incidents of misconduct in recent years, and 17.3% said that they had not had an incident of misconduct. Since this data was subject to an underreporting bias, we did not analyze it. However, these issues warrant further investigation.

Appendix: List of Responding Journals (Using ISI Notation)

• Acoust Phy

• Acs Chem Biol

• Acsms Health Fit J

• Acta Haematol-Basel

• Acta Mech Sinica

• Acta Neurobiol Exp

• Acta Neurol Scand

• Acta Obstet Gyn Scan

• Acta Protozool

• Adv Compos Mater

• Adv Inorg Chem

• Adv Mar Biol

• Adv Synth Catal

• Adv Water Resour

• American Journal of Gastroenterology

• Am J Orthod Dentofac

• Am J Resp Cell Mol

• Am Midl Nat

• Ann Genet-Paris

• Ann Ny Acad Sci

• Annu Rev Control

• Annu Rev Mater Res

• Antarct Sci

• Appl Math Model

• Appl Veg Sci

• Aquat Geochem

• Arch Microbiol

• Asm News

• Atla-Altern Lab Anim

• Aust Meteorol Mag

• Bba-Mol Cell Res

• Behav Ecol Sociobiol

• Biosens Bioelectron

• Biotechnol Appl Bioc

• Biotechnol Bioproc E

• Biotechnol Lett

• Birth-Iss Perinat C

• Bjog-Int J Obstet Gy

• Blood

• Bot Bull Acad Sinica

• Brit J Psychiat

• Can Field Nat

• Can J Anim Sci

• Can J Psychiat

• Carcinogenesis

• Cell Physiol Biochem

• Chem Eng Res Des

• Chem Phys Lipids

• Chirurg

• Classical Quant Grav

• Clin J Pain

• Cns Drugs

• Coal Prep

• Collect Czech Chem C

• Comp Biochem Phys D

• Comput Netw

• Comput Phys Commun

• Connector Specifier

• Curr Opin Clin Nutr

• Diabetic Med

• Discrete Math Theor

• Drug Aging

• Electrochem Solid St

• Environ Entomol

• Eur J Contracep Repr

• Eukaryot Cell

• Evol Comput

• Evol Dev

• Exp Brain Res

• Expert Opin Ther Tar

• Faraday Discuss

• Fungal Divers

• Fuzzy Set Syst

• Gen Physiol Biophys

• Global Ecol Biogeogr

• Helminthologia

• Hematol J

• Hum Factor Ergon Man

• IEEE Pervas Comput

• IEEE T Instrum Meas

• Immunobiology

• Immunol Invest

• Immunol Lett

• Ind Robot

• Inform Sciences

• Int J Antimicrob Ag

• Int J Comput Fluid D

• Int J Mater Prod Tec

• Int J Prod Econ

• Int J Sport Psychol

• Int J Vitam Nutr Res

• Int Polym Proc

• Int Surg

• J Agr Environ Ethic

• J Algorithm

• J Alloy Compd

• J Am Med Inform Assn

• J Anal Toxicol

• J Anim Sci

• J Brazil Chem Soc

• J Chem Eng Jpn

• J Chemotherapy

• J Clean Prod

• J Clin Pharm Ther

• J Comp Neurol

• J Crit Care

• J Dent

• J Dermatol Sci

• J Ecol

• J Ect

• J Endodont

• J Endovasc Ther

• J Eng Educ

• J Epidemiol

• J Food Eng

• J Gerontol A-Biol

• J Hypertens

• J Mech

• J Microlith Microfab

• J Neuroradiology

• J Nurs Care Qual

• J Nurs Scholarship

• J Pediatr Endocr Met

• J Philos Sport

• J Phys Chem Ref Data

• J Polym Eng

• J Porous Media

• J Prosthet Dent

• J Surg Res

• J Toxicol Env Health

• J Web Semant

• Leukemia Lymphoma

• Macromol Rapid Comm

• Mar Pollut Bull

• Math Comput Simulat

• Melanoma Res

• Mol Cell Biol

• Mol Diagn Ther

• Mon Not R Astron Soc

• Nat Phys

• Nat Prod Rep

• Neurotoxicology

• Newsl Stratigr

• Northwest Sci

• Nuklearmed-Nucl Med

• Neuroradiology

• Optometry Vision Sci

• Palaeogeogr Palaeocl

• Paleoceanography

• Pediatr Dermatol

• Pharmacol Rep

• Philos T R Soc B

• Phys Plasmas

• Phys Ther

• Physica C

• Physiol Meas

• Phytocoenologia

• Plant Cell

• Plant Physiol Bioch

• Plant Prod Sci

• Plant Soil

• Plasma Chem Plasma P

• Plos Biol

• Polym Eng Sci

• Primary Care Psychia

• Proteins

• Protist

• Psychiat Res

• Psychol Med

• Public Health Nutr

• Qsar Comb Sci

• Resp Physiol Neurobi

• Rev Mod Phys

• Samj S Afr Med J

• Scand J Clin Lab Inv

• Semin Dialysis

• South J Appl For

• Space Sci Rev

• Spinal Cord

• Stud Geophys Geod

• Stud Mycol

• Supercond Sci Tech

• Surf Coat Tech

• Surg Radiol Anat

• Sydowia

• Synlett

• Terr Atmos Ocean Sci

• Tob Control

• Transport Porous Med

• T Roy Soc Trop Med H

• Ultrason Sonochem

• Vector-Borne Zoonot

• Vie Milieu

• Vision Res

• Weather Forecast

• Wounds

• Zygote