Five-year Report of Croatian Medical Journal’s Research Integrity Editor – Policy, Policing, or Policing Policy

Imagine an action sequence before the opening credits, a combination of an action thriller and a sword-and-sorcery movie:

It is raining outside. That and the fog make this day just another one in a long line of days one should forget as soon as possible. Our righteous “hero,” ie, the Research Integrity Editor, is in his tent, ie, the offices of the Croatian Medical Journal (CMJ). By a signal fire he’s been warned of another foe (a reviewer/co-editor has red-flagged a manuscript, instantly turning it into a “case”). A foe gaining in strength and number (hopefully not), a dark Lord whose forces are in the valley below, hidden by the fog and the rain (obfuscation of actual author contributions). But the hero has lit his torch, ie, computers with access to the journal’s databases and PubMed (1), WAME (2), ORI (3), and COPE (4) websites, sharpened his sword (gathered what little wit he has), and is ready to overcome (well, at least investigate, process, and advise) another foe, another arch-enemy (a possible case of research/publication misconduct). Ah, what a feat! It seems another skirmish is ahead. But greater things are at stake.

Is our hero going to make it? Will righteousness prevail? Will he win the battle? Can it be won? Is there a battle at all? Why is he fighting? For whom? Does he need the fight? Does anyone? Not likely questions for our hero. He pushes on. Relentlessly. Or until the opening credits start. Which is right about… now.

Why start like a movie? For credits, of course. Credits are important. They are important for the stars of a movie. But not only them. They are also important for the rest of the cast, the director, the (associate) producers, the first assistant director, sound designers, supervising art director, set director(s), the director of photography (second unit), stunt coordinators, stuntmen, (post) production manager(s)/supervisor (second unit), key second assistant director(s), script supervisors, (senior, standby and assistant) art directors, 2nd (3rd) assistant director, co-2nd assistant director, additional 3rd assistant directors, (steady)cam operators, 1st (2nd) assistants camera, (key) grip(s), the gaffers, and best-boys. Not to forget the accountants, administrators, technicians, supervisors, designers, carpenters, painters, riggers, electricians, stagehands, propmen, and the whole second unit! What about the hair and make-up artists, security guards, the head of crowd tuition, the composer, the conductor, and the orchestra? Everybody wants credit. The ones that contribute should be awarded credit. Based on their credit they should be rewarded (in the movie industry they are). But how do you reward so many people? How do you know what any of them has done on a specific project? Does the director or the star of the movie contribute more to the movie than the carpenter?

How does all of this apply to a biomedical journal and the biomedical field in general? It mirrors the two extremes in biomedical journals, including the CMJ. Over time the number of authors on the byline has grown considerably (5). In a long byline one may find authors who have contributed very little to the intellectual content of a manuscript (6) as well as the actual hard-working authors who have done most of the work. Why are the former not satisfied with an acknowledgment at the end of the manuscript? The International Committee of Medical Journal Editors (ICMJE) defines authorship criteria as follows: 1) substantial contributions to conception and design, or acquisition of data or analysis and interpretation of data; 2) drafting the article or revising it critically for important intellectual content; and 3) final approval of the version to be published. Authors should meet all three conditions. Acquisition of funding, collection of data, or general supervision of the research group, alone, does not justify authorship (7). Why do some still mistake non-authorship contributions for authorship? The dilution of authorship credit and an inflation in the number of authors may have led to a seemingly lesser importance of authorship, and a lower understanding of authorship credit for the authors (8,9). If authors of a manuscript are not truthful in the disclosure of their authorship contribution, should one doubt the whole manuscript? Many have recognized that trust arises from truthfulness, and that a promotion of ethical rules and guidelines for publication of research is needed, because the problems are widespread (9-16). It seems that constant vigilance and education are vital mechanisms of enforcement of such rules and guidelines.

Three goals

Five years ago, the newly installed CMJ Research Integrity Editor pledged to achieve three goals: promotion of a responsible research culture, development of an Office for Research Integrity, and development of a science-based research agenda (17). In this short Editorial, it is my wish, my duty, and my pleasure to inform you that he and other CMJ editors have been busy fulfilling those promises.

First goal – education

The first goal involved education and training in responsible conduct of research. Through graduate courses, which are mandatory, and postgraduate courses, which are not but are still well received, we have continued the process of education in responsible conduct of research. Using questionnaires, case-studies developed for the courses, and a recently revised edition of the textbook (18) with a newly added chapter covering various aspects of responsible conduct in research, we have started recognizing many of the issues the students, ie, the (future) authors, the editors, and the readers are facing (19,20).

Second goal – National Office of Research Integrity

As reported earlier (17), the Editors of CMJ were involved in the proposition of the Higher Education Act for the Republic of Croatia, passed in 2003 (21). It has taken more than two years for the Act to pass since its original proposal in 2001, and the Article about the governmental body that would promote and regulate research integrity in the Act differs somewhat from the one proposed in the CMJ Editorial five years ago (17). Still, the new Article helped form the Committee for Ethics in Science and Higher Education, a body at the Ministry of Science, Education, and Sports of the Republic of Croatia earlier this year. The Committee’s duties, among others, are the promotion of ethical norms and values in science and higher education and the creation of an Ethics code. One of the things the Committee has already done was the creation of guidelines for responsible conduct for all research grant proposals. In time, the Committee’s role and functionality will, hopefully, reach that of the US Office of Research Integrity and the National Science Foundation Office of Inspector General in the United States of America (3,22).

Third goal – research development

The third goal of development of a science-based research agenda has been addressed very successfully. The CMJ editors have published more than 50 papers in international journals since 2001. Many of the papers deal with publication practices and ethics, attitudes toward science, and knowledge about and adherence to authorship criteria. In a social and scientific setting such as Croatian, where the scientific production (in the field of biomedicine) is low in comparison with other countries (23,24), students (of medicine) are prone to academic dishonesty (eg, cheating and plagiarism) (25,26) and show Machiavellistic behavioral traits (27), and where authors who submit manuscripts to the CMJ do not meet the ICMJE authorship criteria (7) in 60% of the cases (28), the overall outlook for insisting on publication ethics may be bleak.

These practices could be the result of a lack of authors’ knowledge or willful ignorant behavior toward the rules (29), the authors’ unethical behavior, or simply a result of the way the authorship contribution is disclosed (30). We believe the climate of mistrust and leniency toward rule breaking can be influenced and changed by education (19,31,32); hence the idea that we need formal training of our (future) authors and readers. So in addition to our regular graduate and postgraduate courses, the Editors have organized 4 workshops on scientific writing in cooperation with editors of top international general medical journals, the ICMJE annual meeting in Cavtat in 2004, the meeting of editors of Croatian medical journals and editors of international medical journals at the Croatian Academy of Science and Arts in 2004, the international 2-week course on medical statistics for physicians, in cooperation with the University of Cambridge, UK, in 2004, and the First Croatian Workshop on Scientific Publishing – Scientific Journals and Their Editors: Responsibilities and Rights, in 2005, in cooperation with the National University Library (Zagreb, Croatia).

Report

If some people lie, and some people are possible CMJ authors, then some possible CMJ authors could lie. People lie for a multitude of reasons, usually for perceived gains and benefits in their personal, intimate, professional, social, economic, psychological status and well-being (25,33). Some lie because their (micro)societal environment is promoting or encouraging such behavior, or “because it has always been done that way.”

Thus the incentive (and possible reward) to cheat is relatively great. As most of the authors have inadequate formal education in scientific publication, and the pressure to publish is great, it is a wonder that scientific or publication fraud does not happen more often.

I will not try to justify or rationalize cheating, but I will try to show what acts of cheating we caught and how. The authors’ names, topics of their manuscripts, and the circumstances will be concealed as well as possible. The authors might recognize themselves, although I have a feeling they are not going to want to read this editorial. So finally, this here is…

Actual Report

In the last 5 years, 1385 manuscripts have been submitted for possible publication to the CMJ. This is an obvious increase in submissions, comparable to the increase in both the number of articles the journal publishes and the journal’s impact factor. The rejection rate is approximately 60%. The actual case burden was 28 cases of some sort of scientific misconduct, or questionable research or publication practices, which makes about 2% of all the manuscripts submitted. On 4 separate occasions, our Editorial Board was asked to consult other journals or institutions about their cases. Interestingly, out of the 28 cases of some form of misconduct, the cases of questionable authorship contributions or authorship disputes amounted to only 3 cases. This is interesting because junior authors who attend our courses most often complain about problematic and questionable practices regarding authorship – mostly guest or gift authorship, where a “young” author was influenced to add a (usually) senior figure to the byline, and somewhat less with ghost authorship, when a “young” author contributed enough to warrant a place on the byline, but was omitted from it anyway. Obviously, such problems are “solved” prior to the submission of a manuscript. The rest of the cases can be divided into following groups: 10 cases of duplicate or redundant publication, 8 withdrawals, 5 cases of plagiarism (including self-plagiarism), and 2 cases of fabrication and/or falsification (Box). For many of the cases we found several infringements, but filed them as single entities. Here we present just a few illustrative examples.

Fraudulent or questionable research practices

Scientific misconduct of an Editorial Board member. After a long-lasting inquiry and an embarrassing international legal proceeding, one member of our Board was found to have been involved in a case of massive data falsification and fraud, including nondisclosure of major competing interests with the industry. The case took several years to resolve, and the person is now an ex-member of our Board, which is the least of his problems.

The case of missing data and inappropriate communication with the Journal. After an extensive peer-review process, a submitted paper authored by two groups of authors from different institutions received favorable reviews by the referees, but was flagged as problematic by our statistical editor. The editor wanted the authors to re-evaluate some of their data, and present them in a different way. Such a demand first provoked one of the authors to denigrate the statistical editor. When informed such level of communication was unacceptable, the author provoked an explosive, personal altercation with the Editor-in-Chief, finally creating a caustic experience for everyone involved with the journal. Trying to be unbiased, the Editors sent the manuscript out for an external statistical review, and finally to a statistical super-review. The conclusion of all the statistical referees was the data needed re-evaluation and needed to be presented differently, and the conclusions of the research needed to be changed accordingly. Finally, when confronted, the corresponding author inadvertently disclosed they could not re-evaluate the data, because the original data were with the other group of authors where the corresponding author did postdoctoral training. The paper was finally rejected, but the case is still open.

Authorship

“Hurt” gift author. We have recently received a paper for pre-review, to help the authors decide whether it was publication-worthy and to work with them on improving the data presentation, as is routine in the CMJ (32). The Editors thought it was a very interesting and important topic, sent comments to the authors, and worked with them for two months on the Croatian version of the paper, until the paper was considered ready for peer review. Once the translation was complete, the paper was officially received by the Journal and peer-reviewed, and the authorship statements were sent to the authors listed in the byline. Sometime during the peer review process (one reviewer suggested a major and the other a minor revision), the number of authors on the byline increased by one. The paper was accepted for publication, and the Editor, who reads the final version of the article, noticed an extra author on the byline. After checking the authorship statements, we contacted the corresponding author, who maintained that the “added” author contributed significantly from the beginning of research and throughout the writing of the manuscript. The added author was the Head of the Department and an immediate superior to the corresponding author. The failure to inform us about the change on the byline over a period of almost 6 months from receiving the manuscript, over the whole process of pre-review, translation, and the peer-review to the publication, all authors dismissed as a typing error, even though they filled out only three original authorship statements. After further investigation, we concluded that there was a doubt about the contribution of the “added” author and that the authors made a mistake in not communicating the changes to the editors. However, as we consider our role primarily educational, we decided to publish the article after clearly instructing the authors about authorship criteria, providing that the “added” author filled out the authorship statement. A few weeks after the paper was published, the “added” author complained to the Dean of our School about the process and demanded investigation into our conduct.

Publication parasitism. After receiving authorship statements for a manuscript, we discovered discrepancies between the disclosure of involvement in the performance of research and one of the author’s area of expertise. The editors would likely not have noticed such a discrepancy had it not been an author they knew personally. Apart from this fact, the disclosed contributions were almost exclusively non-authorship contributions. When confronted, the corresponding author admitted that the author we were suspicious of, was her sister’s thesis advisor, had nothing to do with the original research or the writing of the manuscript, but needed a publication for an upcoming promotion. An author with such a publication behavior, or rather abusive co-authorship of a senior author, has in a recent publication been aptly termed a “White bull” (34).

Non-disclosed authorship. We received a manuscript for which the authors repeatedly failed (refused) to disclose their contributions. After several months of pleading with the authors, the Editors finally rejected the paper. Interestingly, the authors who had not replied to 4 consecutive e-mail messages about the authorship disclosure, answered the final e-mail informing them of rejection and “chose” to withdraw the manuscript themselves!

Plagiarism

Blatant plagiarism. A clinical research manuscript by 6 authors was found to have had approximately 90% of the Discussion section plagiarized from a paper published 4 years previously in a different journal and by different authors. The list of references of the manuscript revealed that 34 of 36 references were copied from the original article, even in the same order and containing the same typing errors. What was curious about the paper was that the authors went to the trouble of rearranging the rest of the manuscript in order for the references to be transferable between the original and the submitted manuscript. None of the authors wanted to disclose who was responsible, so the case was reported to their Head of the Department (no feedback).

Peer-review abuse

Manuscript withdrawal. Manuscript withdrawal per se is not an infringement of publication or research ethics, but I include these cases in our case-file because the authors usually (ab)used our journal to get an extensive pre-review feedback (32) from the editors and peer-review feedback from our referees, only for us to discover later that the papers were eventually published somewhere else. Even that would still not be a problem, had we been informed that the authors chose to withdraw the manuscript from publication in our journal. For most of the papers, the authors were unresponsive to either our phone calls or e-mails for up to 9 months after having received our referees’ extensive comments, and in some cases even after receiving an acceptance letter from our office.

Simultaneous submission. For most of the cases from the previous group, we have discovered that they were sent to at least two journals for simultaneous peer-review. This was visible by the dates of receipt to other journals once the papers were published. We have contacted the journals to inform them about such publication practices of their authors (no reaction), and consequently will very carefully consider any future submissions from these authors.

Duplicate publication. We received several manuscripts that were exact copies of papers published elsewhere by the same authors. For most of the cases, it turned out the authors were unaware that such publication practice was unacceptable. They mostly claimed that the information the papers were providing, although evidently not novel, was so important it warranted publication in several journals. Some of the authors stated to have done so in the past, explaining to the editors that they had just faithfully followed their superiors’ modus operandi...

Conclusions

Our Editorial Board and the CMJ are not the only ones confronted with problems in the publication process. Rather, we are among the many (35-37). But, in this part of Europe, we were the first to start worrying about it and the first to start doing something about it. Some authors do not like it. Some of those we caught cheating even protested, demanding that we be reprimanded and disciplined. That has only made our determination to educate, promote research, and publication integrity stronger.

But the biggest problems we are facing are basic ones – the problems of principle. As a small journal, with a publication preference for medical topics from developing and emerging countries, and a journal that has only recently begun expanding and gaining international recognition and readership, almost 50% of our authors, or contributors, are still national. Since we are a small nation, we have a limited number of possible authors. Our Editors personally know almost all of our potential national authors, or know of them. Even the ones the Editors do not know personally, they soon will, because the interest in our courses in scientific writing is growing. For this reason, the Editors are usually well-informed about the authors’ overall work, which is advantageous. However, for the same reason, they may be open to possible bias while working with national authors.

So the question we need to ask ourselves is: are we treating national authors more harshly than international authors just because we know their background, or are we more lenient toward them? The other question is whether we need to police our authors. Do we take apart all authorship statements actively looking for any discrepancies? And once such discrepancies or inconsistencies are found, should we take it upon ourselves to dish out justice as we see fit? Or do we make a policy about what infringements to investigate? Do we make it known to all our authors of our new author-unfriendly policy? Who would want to try publishing in our journal for fear of being possibly accused of unethical behavior? Would our future authors be experts in filling out authorship statements, so that the rules are met, at least formally (30)? Or would such strict practices of our Editors lead to a better understanding of the rules and an increase in authorship and publication ethics? A possible question we might ask ourselves would also be – Do we actually need to worry, as only 2% of submissions to our journal were found to be seriously transgressing the international rules and guidelines? The overall rate of 2% of manuscripts not conforming to research and publication ethics is seemingly larger than the 0.2% found in other journals (38), or smaller compared to 7% of serious deficiencies found in drug trials submitted to the Food and Drug Administration (39). But our cases were mostly cases of a minor level of deceit, of questionable research practices, and of author ignorance of publication standards. Would we find more transgressions, and of a more serious nature, if we decided to dig deeper? Knowing that 60% of authors who published papers in CMJ failed to reach full and stringent authorship criteria set by our journal (28) has made us not disclose their contributions in print. Unlike some other journals, we do not actively police our authors, but we still take a similar hard line on these issues once we find serious inconsistencies (37), trying to find out what happened, and why. For now, we are content educating (40) the authors we happen to discover having blatantly transgressed the guidelines. But for how long and to what end is still an open question. Until we reach a final conclusion, we can only hope to raise the level of awareness of ethical issues for everyone and suggest to our future authors to do the following. Ask yourselves three little questions (41): Is it true? Is it fair? Is it wise? Although there is a myriad of possible answers, only one is right. At the beginning of a project, or of writing up a manuscript, these are the three fundamental questions you have to be able to answer to yourself and to everyone involved – be it as a reader, reviewer, or a collaborator/co-author. Should your own answers be affirmative to each of the questions, you are more than likely to be on the right track. But should you wonder, or wander, please do not hesitate to ask.

Box.Croatian Medical Journal’s 5-year experience with research integrity infringements

28 cases (2%) among 1385 received manuscripts:

3 disputable authorship

10 duplicate or redundant publications

8 withdrawals of manuscripts

5 plagiarism

2 fabrication or falsification