Monitoring clinical trials—interim data should be publicly available

Interim results from clinical trials are, by growing convention, scrutinised by committees, commonly called data monitoring committees or institutional review boards. This allows clear evidence of benefit or harm to be identified expeditiously. The UK Medical Research Council sponsored trial of folic acid prophylaxis against recurrence of neural tube defects¹ and a trial of antiarrhythmic medication for prophylaxis against ventricular fibrillation² were terminated early because of favourable and adverse interim results respectively. Current practice is to keep interim data secret, on the presumption that their release would undermine recruitment and provoke “premature” adoption of treatment. Data monitoring committees offer timely expert advice on such matters as data collection³,⁴ and can stop patients being offered randomisation to treatments that would be regarded as inferior by almost any person who had understood the interim data. Knowledge accrues incrementally,⁵ however, and we argue that interim results should be made publicly available, thereby enabling patients to make individual decisions on the basis of data that might rationally provoke different choices from different people. Firstly, we explain why the practice of withholding data is ethically dubious, and, secondly, we argue that making data publicly available has potential practical advantages.

Summary points

• Interim analysis in clinical trials is done in secret because authorities fear premature adoption of promising but “unproved” treatments

• Withholding, without debate and endorsement of the policy, information that patients might find useful is at best paternalistic, at worst authoritarian and arguably unnecessary

• Releasing such information might in fact increase recruitment and would make the task of data monitoring committees easier

“Near term” and “far term” patients

Assume, for the time being, that disclosure of results leads patients to choose a particular treatment and that this leads to failure of further recruitment. Then, these “near term” patients benefit at the expense of “far term” patients (those who would gain in the long term from greater precision contingent on a policy of withholding interim results). As there are, typically, many more far term than near term patients, withholding interim results provides the greatest good for the greatest number.⁶ A data monitoring committee, therefore, has to decide where, between a neutral and strongly positive result, it should act. In doing so, it trades the interests of far term and near term patients as knowledge gradually accumulates.⁷,⁸ Although procedural guidelines exist on how data monitoring committees should operate, no consensus exists on how the interests of near term and far term patients should be traded off or on the principles that should guide this decision.^9–20 Should all potential patients (near term and far term) be weighted equally in a classic utilitarian calculus?⁶ Should overwhelming weight be given to the interests of near term patients? Or is there some compromise position in which near term patients are given some—but not total—preference? The problem is that data monitoring committees make big decisions using opaque (and doubtless variable) heuristics while ignorance is perpetuated to stop potential participants voting with their feet.

Setting up a system that perpetuates ignorance violates Kant's injunction that people should not be used as a “mere” means to an end. Meanwhile, information arising separately from an index trial is not withheld, nor is publication of that information delayed, pending completion of recruitment. Recent guidelines, moreover, charge principal investigators with a duty to keep up to date with the literature and modify the information they give to patients accordingly.²¹,²² So why should data arising in a trial be secret when the same data arising elsewhere would be not only published but actively disseminated? The current practice of withholding data has emerged with no public endorsement, is not based on transparent and replicable methods, and conflicts with other practices. Moreover, this practice may be unnecessary even on its own terms.

Feedback and recruitment

Making data publicly available would enable patients and doctors to take a personal view on the meaning of the interim data. Some—depending on prior beliefs and values—may move into equipoise,⁵ and others may move out. Thus feedback of data would not necessarily always reduce recruitment, and it might even increase it,⁷ while generalisability would probably be enhanced by widening the base of participants. Empirical evidence on the effect of feedback on recruitment is sparse.¹⁴ Publication of interim results of the second international study of infarct survival (ISIS 2)²³ disclosing improved survival of heart attack with clot busting drugs resulted in increased recruitment as initially sceptical clinicians moved into equipoise (R Doll, personal communication). We suspect that if feedback is not available greater public awareness and a perceived threat of censure will provoke data monitoring committees to adopt increasingly risk averse policies, terminating trials in circumstances where informed patients might still have wished to participate.

Feedback trial

In a feedback trial, results could be made available after meetings of the data monitoring committee. Clinicians would then be free to share this information with patients and discuss its limitations and its relation to data arising elsewhere. It could be argued that members of a data monitoring committee would be less likely to over-react to interim data than would the media, which might overinterpret the data. Our view is that this argument derives from a culture of secrecy and scientific imperialism and is self fulfilling: keeping clinicians, patients, and the media in the dark allows naive views to flourish. In addition, bayesian presentation of results could help to prevent overinterpretation, which can occur when data are presented in a frequentist way, since even “significant” results may look unimpressive on bayesian analysis.²⁴

Some trials are concerned with long term treatments—for example, different insulin regimens—where interim data may affect the behaviour of patients receiving ongoing trial treatments, resulting in “crossovers” and dilution of any treatment effect, but we argue that there is an implicit “contract” between researcher and participant in such cases, which makes the obligation to openness all the greater.

Future action

Procedures for monitoring the progress of clinical trials involve ethical, not just statistical, considerations. The topic thus deserves wider debate than it has received hitherto. It may turn out that the public is sanguine about current arrangements or would be content with less radical measures than those we have proposed—for example, inviting patients' advocates to join data monitoring committees and/or deriving a “stopping” algorithm in which value assumptions were made explicit and endorsed by society. Indeed, the consensus might be that current procedures are appropriate in some circumstances (perhaps when study treatments are not otherwise freely available, as in pre-licensing pharmaceutical trials) but that feedback is preferable in others (for example, in very large trials of treatments that are already in widespread use). We believe that the meaning of interim data will vary from person to person, so decisions made on the basis of such data should be individual ones—they should not be collectivised.