Restoring the integrity of the clinical trial evidence base

Abstract

Calling researchers and editors to help restore invisible and abandoned trials

Public confidence in the credibility of medical research is at a low ebb.^{1 2 3 4} Many completed clinical trials have never been published, and many published results are incomplete or misleading.^{5 6 7} This crisis of hidden or misreported information from clinical trials—and the resulting distortion of the clinical evidence base—is widely recognized and commonly decried.⁸ It is one of the leading scientific problems of our time, but few solutions have been put forward.

In a linked Analysis article (doi:10.1136/bmj.f2865), Doshi and colleagues offer a bold remedy in the form of the RIAT (restoring invisible and abandoned trials) proposal.⁹ Invisible trials are those that have never been published. Abandoned trials are unpublished trials that sponsors are no longer actively working to publish or published trials that, although documented as misreported, have not been corrected by the authors. Doshi and colleagues declare that, “because abandonment can lead to false conclusions about effectiveness and safety, we believe that it should be tackled through independent publication and republication of trials.” They challenge medical researchers and funding agencies associated with unpublished or misreported trials to swiftly signal their intent to publish or correct these “abandoned” trials and then to act on this within a year. If no such intention is declared, or if a corrective paper has not been published within a year, they propose offering the opportunity to become “restorative authors” to other responsible researchers, who would restore the integrity of the reporting of the trials involved.

The RIAT proposal outlines the step by step process that the original authors or volunteer restorative authors should follow. It provides a minimum set of criteria for the proper and responsible publication and republication of abandoned studies. To help start this project, the authors of the proposal supply a list of internal company research study reports in their possession; many were obtained as a result of lawsuits or liberalized freedom of information policies. These documents provide detailed, previously confidential, information on a large number of clinical trials that are known to be unpublished or misreported. The authors of the proposal pledge to make these resources available to restorative authors and they call on others with similar holdings to do the same.

As the authors of this proposal explain, it is the existence of clinical study reports that makes it possible to reconstruct industry funded clinical trials. These reports are little known, highly structured internal company documents that describe the planning, execution, and results of individual clinical trials. Why not publish these reports instead of encouraging their distillation into short research reports for journals? These documents may be thousands of pages long and are not easily digestible: journal publication based on them may have a compression factor well above 1000:1.

The authors of the RIAT proposal are confident that the necessary trial information can be obtained from clinical study reports. They provide an audit record tool to ensure that essential information is sorted systematically and to minimize the effect of reporting biases. As well as committing to publication within a year, restorative authors must adhere to the study protocol and its prespecified objectives, as well as to other reporting standards. The aim is to make any value judgments and decisions clear.

Nothing better underscores the urgency and importance of the RIAT proposal than the list of abandoned trials that accompanies it. Read it and weep: on the list are clinical trials for drugs used by millions of people, including zanamivir, atorvastatin, gabapentin, and paroxetine. The number and variety of drugs on the list show clearly that incomplete reporting of clinical trial results is not an isolated occurrence, confined to a few drugs. Rather, it is an entrenched and widespread problem. Secrecy and selective reporting were an integral part of the system. Reforms such as trial registration and mandatory results reporting will improve things in the future but can do nothing about the flawed evidence of the past.

The case in favor of the RIAT proposal is particularly compelling because new treatments are judged against those tested in past trials. If the evidence from past trials is unsound, so will be our view of new treatments. The failure to correct the scientific record is at odds with the principles of transparency that most in the wider medical community, including drug company leaders, now publicly espouse.¹⁰ Despite the rhetoric, however, little has changed so far.

The RIAT proposal is the first to outline a clear practical means to an important end—an accurate understanding of the results of previously performed clinical trials. The proposal authors acknowledge that there are unresolved practical challenges and unforeseen consequences, and many of these challenges were highlighted during peer review of their paper. These problems mean that some will think the project is rash and overly ambitious, whereas others will inevitably think that it does not go far enough. In particular, because clinical study reports exist only for industry funded trials, non-industry funded trials that have been misreported or abandoned by their authors will not find an easy route into the RIAT fold.

We should not let these shortcomings prevent us from moving forward. Doshi and colleagues’ unusual proposal is another step on the road towards a complete and unbiased account of the effectiveness and safety of medical interventions. We hope that the RIAT proposal will stimulate original researchers or capable volunteer restorative authors to come forward. As editors of the BMJ and PLOS Medicine, we endorse the proposal and commit to publishing restorative clinical trial submissions. We encourage other journals to signal their belief in the importance of this effort by endorsing the proposal too, either with an editorial in their journals or by responding to this editorial, encouraging submission of these publications.

The results of clinical trials are a public, not a private, good. The public interest requires that we have a complete view of previously conducted trials and a mechanism to correct the record for inaccurately or unreported trials. If we do not act on this opportunity to refurbish and restore abandoned trials, the medical research community will be failing its moral pact with research participants, patients, and the public. It is time to move from whether to how, and from words to action.

Cite this as: BMJ 2013;346:f3601