Comment on “Adaptive versus fixed designs in confirmatory clinical trials: Centering the choice on ethics”

I identify four important claims in this interesting paper, ¹ which I paraphrase as follows. First, the choice of design is a matter that requires careful consideration and justification. Second, not just scientific but also ethical considerations are important in making that choice. Third, many different parties may need to be consulted to design a trial appropriately. Fourth, we need to be careful in the labels we use in identifying our choices, if we are to avoid talking past each other. In my opinion, all of these apply much more generally than just to the choice between fixed and adaptive designs, as will be illustrated by the example below, in which I shall refer to these four points.

A common view, where a partially effective treatment exists, is that placebo-controlled trials are unethical. The World Medical Association (WMA) Declaration of Helsinki ² (DoH) condemns their use in such cases (with limited exceptions). Similar condemnations have been made by epidemiologists ³ and ethicists. ⁴ However, the development of treatments for human immunodeficiency virus (HIV) infection throughout the 1990s continued to involve trials that were described as placebo-controlled, ⁵ despite the fact that zidovudine (AZT) had been approved in 1987 for treating patients. How was this possible?

The apparent flouting of WMA authority can be explained simply: the WMA misunderstands the nature of placebos, regarding the giving of a placebo, where a treatment is already available, as necessarily involving withholding that treatment. This is not the case. ⁶ Placebos are simply used to blind treatments, and their use may not require the removal of anything. The placebo-controlled HIV trials in the 1990s were add-on trials. Standard of care, including anti-viral treatments, was offered to all patients who were then randomised to receive either the new treatment or a placebo to that treatment in addition. For example, see a 1996 trial ⁷ in which all patients were given zidovudine and didanosine and were then randomised to receive either nevirapine or its placebo.

This is not simply a linguistic quibble (point 4 above). The DoH does both too much and too little.^5,8 Designers of these add-on trials may have considered that the sort of trial that the DoH recommends, comparing new treatments to existing ones, was impractical and unethical (points 1 and 2). Such trials would have denied some of the patients zidovudine and didanosine. How can the WMA get something so obvious so wrong? Possibly by failure to consult widely enough (point 3).

Turning now to the specifics of adaptive versus fixed designs. The authors are surely right that it is unhelpful to award fixed designs gold standard status. Decision theory teaches that a free option is valuable (in expectation), and although, as the authors note, there may be some costs in organising flexibility, this suggests that adaptive designs should always be considered.

Nevertheless, there are (at least) two reasons for caution. The first is that adaptive designs may encourage one to pay too much attention to the accruing data from the trial, ignoring the wider context in which they are placed. This is not an inevitable consequence and cannot be used to dismiss adaptive designs but it is worth being on guard. I wonder if the authors themselves have not fallen into this trap. They write “Specifically, trials are designed based on numerous assumptions—such as expected effect sizes . . .,” ¹ but the delta used in planning should be a function of the disease not the drug, and hence, possible disagreement about delta does not constitute an argument for adaptive studies. Since planned sample sizes are a decreasing function of delta, making delta the effect of the drug would have the unfortunate side-effect of devoting more resources to treatments expected to do badly. ⁹ An alternative view sees clinical trials as being devices (albeit imperfect) to separate the wheat from the chaff. The test they provide is logically independent of the candidate drug, in the same way that university examinations are independent of the candidate student. Of course, this would still leave open the option of early dismissal of candidates for futility.

The second reason is that although the authors rightly argue that sequential designs should be included as a form of adaptive design, there is an important alternative distinction. This is between adaptive designs in which, although many things may change, the allocation ratio cannot and those in which the allocation ratio can. Included in the former are classical sequential designs ¹⁰ and designs using internal pilot studies for sample size re-estimation. ¹¹ Included in the latter are bandit designs ¹² and platform studies. ¹³ The former are not accompanied by any loss of orthogonality, and that means that the efficient analysis is usually unbiased, since concurrent control is observed through all implemented modifications. This is not true for the latter, and efficiency may be adversely affected in consequence. ¹⁴ Adaptive designs have become something of a fashionable mania, ¹⁵ and it is not clear that their adoption has always been thought through.

Nevertheless, I welcome this statement and think that it deserves to make a difference as to how clinical trial design is approached, and hope that it will. Unfortunately, experience gives no reason to be optimistic. We have known for decades that dichotomisation is unscientific and, by Doug Altman’s criterion of wasted research, ¹⁶ unethical. Yet, this wasteful habit ¹⁷ continues to be defended in terms of the completely specious argument of clinical relevance. Calling for a rational and ethical approach to trial design does not solve the problem of establishing what that approach entails, still less ensure consensus as to what should be done.