Investigating the previous studies of a fraudulent author

Short abstract

This year, the journal Nutrition retracted a study by R K Chandra, and questions have been raised about the integrity of the rest of his work. Who has the responsibility for investigating previous work and if necessary punishing the researcher and correcting the scientific record?

In February of this year, Michael Meguid, the editor of Nutrition, retracted a paper by the Canadian researcher R K Chandra, that it had published in 2001.¹,² The paper claimed to be a randomised double blind placebo controlled trial showing that physiological amounts of vitamins and trace elements would improve cognitive function in elderly people.¹ Meguid gave eight reasons for retracting the paper and said that Chandra had either ignored the reasons or failed to give an adequate response.²

Chandra's paper was submitted originally in 2000 to the BMJ, which had severe doubts about the paper: one reviewer said that the paper “had all the hallmarks of having been entirely invented.”³ The BMJ asked Chandra's employers—the Memorial University of Newfoundland—to investigate its anxieties about the study. The university held an inquiry but found no serious problem. The BMJ was unconvinced by this response and raised further questions about the study. In August 2002 the university answered that Chandra had taken unpaid leave for the first four months of 2002 and failed to respond to any of its inquiries, including a request for raw data. Then in August 2002 he resigned.

Meanwhile, the BMJ had notified Nutrition about its anxieties over the study. Unfortunately Nutrition had already published the study. Chandra must have sent the study to Nutrition as soon as the BMJ began questioning it. The BMJ also notified the Lancet, which had published a closely related study by Chandra in 1992.⁴ Serious doubts were then raised about the 1992 study in a letter to the Lancet in 2003, which among other criticisms pointed out that “some of the standard errors were statistically impossible.”⁵ There must be grave doubts about the Lancet study, which has been cited more than 300 times,⁶ and about the other 200 papers published by Chandra, many of which are randomised trials with him as sole author. Furthermore, investigations by the Canadian Broadcasting Corporation have raised many other questions over the integrity of Chandra, who is an officer of the Order of Canada and holds a patent for the supplement that is claimed to improve cognition.

Figure 1.

R K Chandra's paper in Nutrition was retracted; he resigned before the Memorial University of Newfoundland could investigate his previous studies

Credit: DEAN PARSONS/www.groundeffectimaging.com

The Memorial University of Newfoundland has said that it can do no more about investigating Chandra and that responsibility for investigation lies with the journals that have published the more than 200 articles. The Canadian Institutes of Health Research have tried to hold an investigation, but Chandra has refused to give research data.⁶

This sad story raises fundamental questions about who has the responsibility for investigating allegations of research fraud and particularly for considering other studies published by a researcher who has been shown to have written one fraudulent paper.

Investigating previous studies is important

When the fraud squad discovers that somebody is guilty of an act of financial fraud they assume—until proved otherwise—that much else in the person's affairs is fraudulent. Yet within medicine we seem to be inclined to believe the opposite: a fraudulent research paper is often seen as a one off caused by the author experiencing stress, mental illness, or difficult circumstances. It is perhaps to be expected that policemen, who spend their lives associating with criminals, will assume the worst, and doctors, who are agents of mercy, the best. But it is clearly dangerous when we discover a fraudulent research paper to assume that there were no problems with all previous work. The safe—and right—thing to do is to assume the opposite and start an investigation.

Patterns of misconduct

Stephen Lock's “imperfect history” of research misconduct in medicine shows both how cases of fraud are likely to be seen as isolated incidents but how investigations often eventually show a history of previous fraud.⁷ The modern story of fraud, writes Lock, began in 1974 with William Summerlin, from the Sloan-Kettering Institute in New York, claiming to have transplanted human corneas into rabbits. He also faked transplantation experiments in white mice by blackening patches of their skin with a pen, an extraordinarily crude form of forgery. Summerlin's misconduct was long ignored and once exposed was attributed to a mental health problem.

John Darsee worked in the department of cardiology at Harvard and was seen falsifying data. His boss, Eugene Braunwald, an eminent cardiologist, decided that this misconduct was an isolated incident and so did not fire him. A few months later, however, it became clear that results he had taken in a study being conducted in several places were very different from those of the others. An investigation was started and went back to when he was an undergraduate. Many of his more than a 100 studies proved to be fraudulent and had to be retracted. Darsee's high profile case led to congressional hearings. And Drummond Rennie and Kristina Gunsalus have described how the first witness, the president of the National Academy of Sciences, “asserted that problems [of misconduct] were rare—the product of `psychopathic behaviour' originating in `temporarily deranged' minds.”⁸ Unfortunately, accumulating evidence shows both that they are not rare and are usually a pattern of behaviour rather than isolated cases.⁹,¹⁰

Who should investigate?

An accusation of research misconduct is clearly a serious event. Such an accusation cannot be ignored, but nor can the accusation be assumed to be correct. There needs to be a process of screening the accusation, investigating it if necessary, conducting a hearing, making a judgment, and taking whatever steps may be appropriate to punish the offender, to make changes to reduce the chance of misconduct recurring, and to correct the record. The process must be effective, fair, and efficient, and it has to be conducted by a body that has the legal legitimacy to do so and the means to administer suitable punishment. Often accusations are about a particular piece of research, and my argument is that there needs to be a secondary process of investigating previous work. The body that has responsibility for the investigation of other research might also want to examine financial and professional matters.

The investigation of an accusation can be difficult, but I must acknowledge that the investigation of previous work is still harder. Examination of published papers will rarely be adequate for deciding confidently whether work was fraudulent. The research may have been done 20 years previously—and so neither raw data nor records may be available. Possible witnesses may be dead or dispersed and may have to try and remember events from long ago. Miscreants may have published hundreds of papers, and there may be vast quantities of data to examine. Unsurprisingly, institutions may try to avoid having to investigate previous work

Employers

One body that clearly has legal legitimacy to respond to an accusation of research misconduct is the accused's employer, usually a university, a medical school, or a hospital, in the case of a medical researcher. Increasingly these institutions do have declared, transparent, and legally sanctioned processes for dealing with accusations of research misconduct—not least because the funders of research require such processes to be in place. But until recently, many institutions did not have satisfactory processes, and globally it is still probably the case that many, even most, do not. In many countries the problem of research misconduct has not been confronted. Furthermore, every step of the process is important, and my suspicion is that it is a minority of employers that have processes that work well at every stage. The investigation and collection of evidence, for example, is something that is unfamiliar to many institutions, which is why they may have to hire an agency to do the work for them.

Employers face a clear conflict of interest with accusations. To expose one of your employees as fraudulent is unpleasant and does harm to the brand of the institution. Often the miscreant will be a friend, a respected colleague. The temptation is to bury the whole thing, and Stephen Lock has produced evidence that this did happen commonly.¹¹ Many, even most, senior academics know of cases of misconduct, and yet they report that rarely were those cases fully investigated, the miscreant punished, and the scientific record corrected. The miscreant may simply have been encouraged to resign or take early retirement. In such circumstances, the scientific record is not corrected—and certainly there is no investigation of other research studies by the miscreant.

As well as being unpleasant for employers these cases can also be expensive, time consuming, and difficult. (I remember ringing a hospital in the United Kingdom with an anxiety that one of its consultants may have published a fraudulent study in the BMJ, which I edited at the time. The hospital told me that the consultant was already suspended because of anxieties over his clinical work and that if the BMJ wanted an investigation of his research it would have to pay.) Miscreants may well employ lawyers and challenge the processes of the employers. Costs and complexity immediately increase. Or the miscreant may resign—so removing the legal legitimacy of the employer to investigate. Or, commonly, the miscreants may come from several institutions, often in different countries, making the organisation of an investigation and hearing exceedingly complex.

Institutions have great difficulty investigating accusations against their employees—and attempts often fizzle out. They do, however, in most countries currently have more responsibility, capacity, and legal legitimacy for conducting investigations and correcting the scientific record than any other institution—as emerges in what follows as I examine alternatives.

Funders of research

Funders of research will have a contract with those they fund, and this contract will give them a legal basis for investigating an accusation of research misconduct in relation to the work that they have funded. They will not, however, have legal legitimacy for investigating research that they have not funded, and funders of research usually require employers to investigate accusations of fraud rather than attempt to do it themselves.

Regulatory bodies

Some professional regulatory bodies may respond to accusations of research misconduct, and the General Medical Council (GMC) in the United Kingdom, for example, has dealt with many cases.¹² Most of these have been brought to the council by pharmaceutical companies, and most had already been investigated by a professional team that included a former policeman.¹²

One reason that bodies such as the GMC cannot provide a full answer to the problem of who should respond to accusations of fraud is that they usually cover only particular groups—the medically qualified in the case of the GMC. And most medical research is done by people who are not medically qualified. Other problems are that such bodies usually deal with individuals not teams and consider particular cases and episodes.

Colleges and professional societies

Professional societies—such as the Royal College of Physicians or the European League Against Rheumatism—might seem to be obvious bodies to investigate research misconduct within medicine, but usually they do not have either the legal legitimacy or the means. Furthermore, the worst they can usually do in terms of punishment is expel the miscreant.

Journals

It is journals that publish the studies that may be deemed fraudulent, so are they not the obvious bodies to investigate accusations? Unfortunately, they cannot. The fact that an author has published a study in a journal does not give the journal the legal legitimacy to investigate even that particular study. Nor can a journal apply due process. A journal making a judgment on whether a paper was fraudulent would be nothing more than “trial by media.” The journal clearly, however, has a duty to notify its readers if the paper proves to be fraudulent. It must depend on others, usually employers, to hold an investigation and hearing and reach a conclusion on the status of the work.

Journals are privileged whistleblowers—privileged because they are hard to attack and whistleblowers because they, their reviewers, or readers are often the first to suspect research misconduct. Reviewers or readers might choose to contact employers rather than journals, but they rarely do so—perhaps because as individuals they are more vulnerable than journals. The job of the journals is to ask another body—usually an employer—to investigate. But they also have a duty to make sure that the responsible body stays with the case. If all else fails (as, sadly, it does) then the journal may have to “tell the whole story,” but their lawyers will advise them against it unless they have undertaken an unusually thorough investigation.

One difficulty for the journals is to decide the amount of evidence that is necessary to blow the whistle. It is best not to spend too much time building up evidence but rather to quickly transfer the responsibility for investigation to the employer. When as an editor I first dealt with possible cases of fraud, I spent a lot of time gathering evidence, but this backfired on me on at least one occasion when the story leaked out. Journals can ask for access to the raw data behind a study, but analysing raw data is by no means straightforward. It requires special skills and can be expensive.

I applaud Michael Meguid for working so hard to question the study of Chandra published in his journal Nutrition, and the BMJ is in this issue publishing a paper that describes doubts over another researcher that go back at least 10 years.¹³ But we are left not knowing the true status of large numbers of research papers by these authors.

The criminal justice system

Alexander McCall-Smith has made the interesting suggestion that the criminal law might be used to manage research misconduct.¹⁴ Such cases will often involve misuse of funds—perhaps grants from public bodies—and so research fraud may be seen as an example of financial fraud. I do not know of this having happened, and the immediate reaction of most academics and researchers is that this would be an extreme step. But the great advantage of such a development would be that the criminal justice system is used to dealing every day with problems that often overwhelm academic institutions.

National bodies

Some countries—including the United States and the Scandinavian countries—have national bodies that can deal with accusations of research misconduct. They may either conduct the whole process or advise and oversee employers. They have legal legitimacy, established processes, and growing experience, and where they exist they may be the obvious organisation not only to lead on particular cases but also to take on the difficult task of examining previous studies.

Summary points

Once a piece of work is proved to be fraudulent, the integrity of other studies by the author should be assessed

An effective, fair, and efficient process to investigate and if necessary, hold a hearing, mete out punishment, and correct the scientific record is essential

Employers are best able to conduct this process, but it is not easy—particularly with studies conducted years ago

Because employers and others often cannot cope and because many studies include authors from many countries there may need to be an international body to take the lead

A mechanism for marking studies as “dubious” on databases such as PubMed is necessary because it is often impossible to reach a confident conclusion on the integrity of previous studies

Unfortunately, such bodies do not exist in most countries, including the countries of the United Kingdom. They can also have difficulty dealing with cases where the possible miscreants come from several different countries, which is increasingly the case in research.

The media

The media should not be the bodies to investigate accusations of research misconduct, but they often are—because of the failure of formal legally legitimate bodies. This seems to be the case with Chandra, where the Canadian Broadcasting Corporation has taken the lead.

An international body

Ideally—because medical research is increasingly an international activity—an international body could take the lead on responding to particular accusations of research misconduct and investigating previous research studies. Establishing the legal legitimacy of such a body would take time, and there would be inevitable arguments over funding, governance, and geographical positioning. But such a body could potentially overcome the problems that rule out most other possible contenders for the job of responding to accusations of research misconduct. Interestingly, physicists—who have come later than medical researchers to the problem of research misconduct—have started with the assumption that they need an international body.

Should we mark suspicious studies?

One of the most important reasons for investigating all the studies of fraudulent researchers is to correct the scientific record. Ideally fraudulent studies will be retracted and so excluded from the systematic reviews that are becoming ever more important sources of evidence. It may well be, however, that it will be impossible to work out the true status of many studies, particularly those done long ago. Should we therefore have a system of marking such studies as “dubious”?

Conclusions

This article has arisen from a dispute between the BMJ and Memorial University of Newfoundland over who has responsibility for investigating accusations of research misconduct against Chandra. In the absence of national or international bodies, I believe that the responsibility lies with the Memorial University of Newfoundland. It certainly does not lie with the BMJ, which blew the whistle on one of his papers (without publishing it) and published only a couple of short pieces from Chandra a long time ago.

Ideally, however, there would be an international body to take the lead—on this and other cases that are left unaddressed. Furthermore, although we lack effective mechanisms for dealing with such cases we perhaps need a means of marking studies that are under suspicion in databases such as PubMed.