Trial protocols: time for more than tinkering

Editor—The BMJ's recent requirement that authors include a trial protocol when submitting research papers for publication seems reasonable to expose deviations from the original protocol.¹ However, Abbasi and Jones's comment that “identifying deviation from the protocol is another important step in ensuring that the findings of a study are reported with honesty and transparency” rings hollow.

What are the other “important steps”? Is there, for instance, any evidence that such steps taken in the past have made a genuine difference to the reliability of data from clinical trials? Despite the many attempts to address the poor quality of reporting of trials, studies suggest that there has been little improvement in the standard.²,³

In any case, compelling researchers to include their protocols will have no effect on their honesty. If people are dishonest and intent on committing research misconduct, then they will do so—preventing them from deviating from the protocol will simply encourage them to seek other ways to achieve their aims. And, it is not difficult to discover many instances of investigators being willing to manipulate data in other ways.⁴,⁵

The point is surely this: current clinical research relies on a methodology that is readily open to abuse. The mechanics of large scale randomised trials are highly complex, the data opaque, and the results marginal—all features that prevent a clear view of any research misconduct. Moreover, the absence of any method to confirm or refute the results means that such behaviour is likely to go undetected.

Dishonest researchers exist and randomised clinical trials allow them to flourish. Tinkering with new rules—such as that proposed in the editorial—will do little to improve the reliability of the results of clinical trials. What is needed is a complete reappraisal of large scale randomised trials.