Reply to Villar et al

To the Editor—We thank Villar et al for their commentary on our article [1, 2]. We found their references informative. Their first point is that we overgeneralized because response-adaptive randomization (RAR) includes power-based and other methodologies with greater robustness to the concerns that we raised. We agree with this assessment. Our most pressing concern is about the methods being proposed and used in actual clinical trials, as purveyors of RAR do not appear to be using power-based approaches. This may be because power-based thinking seems counter to the intent of RAR. Many phase 3 trials use serious clinical endpoints such as death, whose probability of occurrence is much lower than 0.5; in that case, power is maximized by assigning more patients to the arm with the higher event rate! Incidentally, others have also generalized about RAR: “Many variants of RAR have been proposed in the literature. However, different RAR procedures often perform similarly because they obey the same fundamental principle” [3].

Temporal trends remain a major concern regardless of methodological approach, particularly in infectious diseases. Villar et al suggest that power-based methods are robust to time trends and that the likely magnitude of a time trend is an important consideration. The magnitude of a time trend is hard to predict but can be large in platform trials of infectious diseases, for example, for the reasons cited. There is little solace in knowing that the bias and other damage caused by RAR might be “not too bad,” given that we can eliminate bias from temporal trends using standard randomization methods.

We agree that asymptotic results suggest that standard methods can be used when there are no temporal trends, and continue to believe that only a re-randomization test protects against arbitrary temporal trends.

We agree that some RAR methods have low probability of assigning more patients to the worse arm, an improvement learned only after missteps. We ponder how many other problems might be lurking for a method that has been seldom used despite existing for more than 80 years.

A clinical trial is the gold standard for quantifying the effects of interventions. Randomization is the essence of a clinical trial. If randomization fails, then the trial fails. With conventional randomization methods, temporal trends equally affect the treatment and control arms. This is not true with RAR. It is a disservice not to appropriately educate researchers and research consumers about this fact, particularly in the context of infectious diseases and in the midst of a deadly pandemic, where correct answers are crucial. We agree that not all RAR methods are the same and that improvements have been made to RAR methodologies through lessons like the Extracorporeal Membrane Oxygenation trial. Those improvements, such as a “burn-in” period of conventional randomization, have moved the originally proposed RAR closer to conventional randomization. Continue moving in that direction ad infinitum and we will eventually agree.

Note

Potential conflicts of interest. M.P. has no potential conflicts to disclose. S.E. reports personal fees from Takeda/Millennium, personal fees from Pfizer, personal fees from Roche, personal fees from Novartis, personal fees from Achaogen, personal fees from Huntington’s Study Group, personal fees from ACTTION, personal fees from Genentech, personal fees from Amgen, personal fees from GSK, personal fees from American Statistical Association, personal fees from FDA, personal fees from Osaka University, personal fees from National Cerebral and Cardiovascular Center of Japan, personal fees from NIH, personal fees from Society for Clinical Trials, personal fees from Statistical Communications in Infectious Diseases (De Gruyter), personal fees from AstraZeneca, personal fees from Teva, personal fees from Austrian Breast & Colorectal Cancer Study Group (ABCSG)/Breast International Group (BIG) and the Alliance Foundation Trials (AFT), personal fees from Zeiss, personal fees from Dexcom, personal fees from American Society for Microbiology, personal fees from Taylor & Francis, personal fees from Claret Medical, personal fees from Vir, personal fees from Arrevus, personal fees from Five Prime, personal fees from Shire, personal fees from Alexion, personal fees from Gilead, personal fees from Spark, personal fees from Clinical Trials Transformation Initiative, personal fees from Nuvelution, personal fees from Tracon, personal fees from Deming Conference, personal fees from Antimicrobial Resistance and Stewardship Conference, personal fees from World Antimicrobial Congress, personal fees from WAVE, personal fees from Advantagene, personal fees from Braeburn, personal fees from Cardinal Health, personal fees from Lipocine, personal fees from Microbiotix, personal fees from Stryker, grants from NIAID/NIH, outside the submitted work. Both authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest.