Innovative Trial Designs to Improving Tuberculosis Drug Development

To the Editor—The article by Phillips et al [1] outlines pressing needs in the development and evaluation of novel drug regimens to curb the global tuberculosis epidemic. Innovative trial designs are needed to improve the pace at which more effective, less toxic, and shorter regimens become available. Ideas presented in this article have considerable appeal in this regard, and the adaptive multiarm multistage (MAMS) design is a promising phase II strategy for selecting treatments to be evaluated in phase III trials. We are concerned, however, that the presentation of the MAMS design can too easily be misinterpreted. The following statement is a case in point [1pS251–2]:

“A major benefit is that standard statistical techniques can be used to compare the control with those regimens that are not eliminated at interim analyses without any need for complex correction for bias. This is in contrast to a ‘pick-the-winner approach’…where only the best-performing arm (or arms) is selected at the interim analysis to be taken forward …. Because the MAMS design involves the dropping of poorly performing arms rather than the selecting of the best arms, the estimates in the arms that are continued remain unbiased. The estimates in the arms that are dropped are not unbiased.”

The passage quoted above implies that, when dropping arms, no statistical adjustment is necessary to maintain the targeted type I error rate and that estimates of treatment effect remain unbiased. Although elimination of poorly performing drugs rather than picking the best drug would seem to produce less inflation of the type I error rate, most ways of dropping poorly performing drugs have a potential for inflation. The devil is in the details of how one appropriately ensures control of the type I error rate: not all approaches will control it, and the level of control required depends on the phase of development. Failure to take effective treatments forward to confirmatory evaluation would be a serious error in a phase II trial, while strict control of the probability of sending inactive drugs forward (ie, the type I error rate) is a lesser concern. In a phase III trial, however, the emphasis shifts to ensuring a low probability of making any errors; high power and control of the family-wise error rate are paramount. Under a phase III design that drops poorly performing arms, control of the probability of 1 or more errors is important to ensure that ineffective therapies do not become accepted. Further work evaluating the operating characteristics of designs such as the MAMS is necessary to assure this.

In summary, we commend the authors for highlighting important issues facing drug development in tuberculosis and encourage clinical trialists to further develop designs and analysis methods to improve the speed of drug development in order to address this important public health need.