Doubts over head injury studies

Abstract

Patients are receiving treatment that may be unsound as investigations by Ian Roberts and colleagues raise questions about whether influential trials of high dose mannitol ever took place

Each year, worldwide, many thousands of people are treated in emergency departments for head injuries. Mannitol is an osmotic diuretic that is believed to reduce intracranial pressure after head injury and may improve patient outcome. Between 2001 and 2004, a Brazilian neurosurgeon Julio Cruz and colleagues published three clinical trials comparing high dose and conventional dose mannitol in the treatment of head injury (table).¹,²,³ No other trials had examined this question.

 Details of trials of high dose mannitol

Authors	Patients	Intervention	Outcome	Results		Odds ratio (P value)
				High dose	Low dose
Cruz J, Minoja G, Okuchi K1	178 adults with non-missile, traumatic acute subdural haematomas	High dose mannitol v lower dose mannitol	Death at 6 months	13/91	22/87	0.49 (0.06)
			Death or severe disability at 6 months	28/91	47/87	0.38 (0.002)
Cruz J, Minoja G, Okuchi K2	141 patients with traumatic, non-missile, acute, intraparenchymal temporal lobe haemorrhages associated with early abnormal pupillary widening	High dose mannitol v lower dose mannitol	Death at 6 months	14/72	25/69	0.42 (0.03)
			Death or severe disability at 6 months	28/72	46/69	0.32 (0.001)
Cruz J, Minoja G, Okuchi K, Facco E3	44 adults with traumatic, non-missile, acute, severe, diffuse brain swelling with recent clinical signs of impending brain death	Very early and fast high dose mannitol v lower dose mannitol	Death at 6 months	9/23	14/21	0.32 (0.07)
			Death or severe disability at 6 months	13/23	19/21	0.14 (0.01)

The results showed that high dose mannitol greatly reduced death and disability six months after the head injury. A Cochrane systematic review that included these trials concluded: “high dose mannitol seems to be preferable to conventional dose mannitol in the acute management of comatose patients with severe head injury.”⁴ However, one of the trials was accompanied by an editorial that questioned the reliability and validity of the results, calling for further multicentre studies.⁵ A subsequent investigation by the Cochrane Collaboration was unable to confirm that the studies took place.

Doubts over the data

In May 2006, Dr Jorge Mejia, the Colombian national coordinator of the CRASH-2 (clinical randomisation of an antifibrinolytic in significant haemorrhage; www.crash2.Lshtm.ac.uk) trial, wrote to IR (who is editor of the Cochrane Injuries Group) after attending a meeting of the Latin American Brain Injury Consortium in Brazil. He was concerned about the inclusion of the Cruz trials in the Cochrane review:

“During the discussion some Brazilian physicians expressed some surprise with the inclusion of Julio Cruz' paper in the meta-analysis (Cruz 2004; J Neurosurgery, 100:376) … Cruz had no patients at his arrival to Brasil, back from USA where he had developed his research career.”

Dr Mejia was clearly shocked by this revelation: “I do not know what to do, but I feel betrayed. I guess that someone should contact the others authors and ask them. I feel that I can not stay passive, but I have no evidence.”

Dr Cruz, the lead author, had killed himself in 2005. However, the reports had coauthors from Italy and Japan, and we contacted them for further information.

Dr Minoja wrote on 15 May 2006:

“It was a pleasure for me to be included along the authors of the papers written by Julio Cruz on the use of high doses of mannitol in head trauma emergencies. My Unit and I did not provide to Dr. Cruz a personal series of randomized patients, but my contribution was discussing with him and sharing his assumption, because occasionally, in emergency situations, I have used and I still use with success aggressive high-dose mannitol approach.”

In a second message, he wrote: “I think that patients were enrolled in USA and more recently in Brazil, but I honestly don't know the period and in what Institution.”

Dr Okuchi responded:

“Since I did not conduct any study related to the results of Dr Cruz's high-dose mannitol trials in Japan, I have no data to present you. I did not know any part of the paper before he called me about the acceptance in the journal every time” (18 May 2006).

Dr Facco wrote:

“The paper I am co-author, springs from the clinical experience I shared with Julio Cruz about the potential effectiveness of high doses of mannitol in selected very critical patients. I discussed with him (mainly by phone) about my anecdotal experience I never had the opportunity to check in a prospective study, and he also had the same clinical impression of its effectiveness .... Following our discussion, he decided to test high mannitol doses in the emergency setting and involved me as co-author, but my role was “philosophical” rather than clinical: to my knowledge, the study was conducted personally by Julio, probably in Brazil, and I only helped him with discussion and text revision” (22 May 2006).

Since none of the authors could provide any reassurance about the integrity of data, the Cochrane systematic review was withdrawn in 2006 while an investigation was made.

The Cruz papers were published in Neurosurgery and the Journal of Neurosurgery and we wrote to the editors about the concerns. In July 2006, John Jane, the editor of the Journal of Neurosurgery wrote:

“I have tried unsuccessfully to contact you by phone with regard to the Cruz papers. As you can tell by Dr. Marshall's editorial, we all doubted the data. But to doubt is different from concluding that Dr. Cruz fabricated the data. I thought he did, but hoped as stated in the editorial that publication would encourage repetition of the studies. My Editorial Board thought Dr. Cruz' work should be published. I wouldn't trust the data.”

The editorial by Dr Marshall which accompanied the Journal of Neurosurgery report stated:

“These results are clearly of substantial interest, but also raise questions about how reliable and valid are clinical studies that show very dramatic improvements in outcome when they are performed at only one institution. This does not demean the work of Dr Cruz and his colleagues; rather it indicates that multicentre studies, such as those being conducted at present for novel pharmacological therapies, need to be applied to alternative dosing regimens for more traditional methodologies.”⁵

We asked Dr Marshall if he had any concerns about the integrity of the data. He would not respond in writing but he left a phone message saying that he had “serious concerns” about the paper (8 August 2006).

The editorial office of Neurosurgery wrote:

“With a serious charge, the possible fabrication of trials, a case must formally be presented with any and all possible evidence that would indicate the cause for concern and the reviewers must then address the issue. It is not possible or responsible for the Editor of a peer-reviewed journal to act hastily in such a matter without the input of the reviewers who originally accepted the paper, nor would it be responsible to simply pull the papers without presentation of the results of an investigation into the matter. An unsubstantiated claim and verbal recitation of the inquiries made cannot suffice” (8 December 2006).

We have had no further correspondence.

Where did the patients come from?

None of the reports indicated where the patients came from. All of them stated that “institutional review board approval was obtained” but gave no further details. On each report, the Dr Cruz's affiliation was the Comprehensive International Centre for Neuroemergencies and Federal University of São Paulo.” Reprint requests were to the centre at a postal box address in São Paulo. We were unable to find any further information about the centre. In October 2006, the Federal University of São Paulo stated in response to our inquiry that it had never employed Dr Cruz.

We also wrote to the Brazilian national committee on ethics in research, which began an investigation. The investigation found that Dr Cruz had given an interview in which he said that the patients in one of the trials were from eight hospitals in Brazil, Italy, and Japan. However, the authors from Italy and Japan had told us that they did not enrol any patients at their hospitals. Dr Minoja had said that he thought that the patients might have been recruited in the US (15 May 2006). We contacted the University of Pennsylvania, where Dr Cruz had worked until March 1995. They searched their records but found no indication that the research was conducted there (Steven Fluharty, personal communication, 8 December 2006).

In September the Committee on Publication Ethics recommended that the living authors seek retraction of the reports on the basis that they were gift authors and could not take responsibility for the results. We wrote to all three coauthors on 21 September asking if they would be willing to seek retraction.

Dr Okuchi replied the next day: “I would like to retract these papers from the journals because I am not able to take responsibility for the content. Could you let me know how to act formally for the purpose.” A few days later (25 September) he wrote again, “On my last e-mail letter, I had mentioned withdrawal of the papers, however I found that I had no right to retract the papers from the journals. I think it depends on whether Dr Cruz will decide or not ... I will contact him in a few days.”

IR wrote again to the coauthors on 10 October explaining that Dr Cruz was dead, stating that “it would be wrong of me not to follow up this matter and so unless you contact the journals in question to seek retraction I will have to write to your institutions to ask for their help. It is an unfortunate business but it will be better for everyone concerned that you take the appropriate steps to resolve this matter.”

Later that month (27 October), Dr Facco wrote on behalf of Dr Minoja and Dr Okuchi. He said that they did not believe that they were gift authors and declined to seek retraction. He argued that the papers were published in an international peer reviewed journal, that the first author had taken responsibility for their content, and that they knew the first author well and believed that “he would never have been able to do something false.”

Wider implications

We are left with serious doubt about important studies but with no way of determining with confidence whether the results are fabricated or real. The main author is dead. There is no institution to investigate. The implications for patients are serious. They are being treated on the basis of potentially unreliable evidence. It is plausible that mannitol in high doses may increase rather than decrease brain swelling.⁶ Shortly after the withdrawal of the Cochrane review, the Cochrane Injuries Group was contacted by US researchers preparing guidelines for the management of severe traumatic brain injury and by BMJClinical Evidence asking about the outcome of our investigation because the Cruz results were about to be incorporated into guidelines.

There are also implications for the broader scientific community. Earlier this year an investigation by Science of more than a dozen fraud or suspected fraud cases showed “uneven and often chaotic efforts to correct the scientific literature.”⁷ If it wants to retain the confidence of the public and politicians, the scientific community needs to do better. Only a minority of countries have an effective national system for responding to scientific misconduct. However, research is a global enterprise and a strong case exists for an international body to respond to the problem of research misconduct.