Introduction
Randomised controlled trials (RCTs) are the gold standard for a comparative evaluation of interventions. Their robust design helps prevent different biases, most importantly confounding by indication. However, RCTs often require large numbers of patients, and even then many appear to be underpowered—and thus inconclusive—due to misspecification of original assumptions used for sample size calculation [1, 2]. Furthermore, especially in critically ill patients, it is difficult to acquire informed consent for interventions that need to start immediately, such as treatment of infections. This may result in selected populations, reducing the generalisability of study findings [3]. Adaptive trials are trials that include decision rules to change key trial design elements during the RCT. The promise of adaptive trials is to provide answers to therapeutic research questions as efficiently as possible without compromising reliability. They can be designed such that a conclusive answer is always reached and that—during the course of the study—the proportion of patients receiving the most promising treatment increases [4]. This benefit for individual patients may overcome ethical barriers to apply deferred or waived consent for randomisation, and thereby increase generalisability of the results. In this viewpoint we aim to elucidate principles, advantages and pitfalls of adaptive trials.
The first adaptive trials were performed in the 1970s, but were not widely adopted due to methodological shortcomings, lack of understanding by clinical investigators, and ethical concerns about weighted randomisation [5]. To the best of our knowledge, in critically ill patients only five adaptive trials have been performed (all using adaptive sample sizes [6–10]) and one is ongoing (ClinicalTrials.gov NCT02735707). As recent improvements now overcome most of the methodological and technological shortcomings, adaptive designs are gaining more attention [11].
What is an adaptive trial?
Key trial design elements that could be subject to adaptation during the RCT are (1) sample size, (2) intervention arms, (3) allocation ratio, and (4) study population (Table 1). As a result, adaptive trials will—upfront—always have an unknown sample size. Importantly, adaptive trials do not provide a free ticket for trial adaptations: adaptations are based on the analyses of accumulating data with adaptation rules being pre-specified in the study protocol.
Table 1.
Design element | Adaptation | Advantages | Risks and challenges |
---|---|---|---|
Sample size | Stopping rule if superiority is met. May also include stopping rule for equivalence | Efficiency/utility: Sample size determined by true rather than assumed effect size. Study can be designed such that it always reaches a conclusion Ethical advantage not to expose patients to experimental treatments if conclusion is already reached and not to enrol patients into a trial that has a fair chance not to reach a conclusion |
Study may become larger than expected due to small effect size or by chance. Funding agents may be reluctant to support studies with unknown sample size |
Interventions | Drop inferior intervention arms. May also include rules to add intervention arms | Ethical advantage: less patients randomised to inferior arms Statistical efficiency: more patients randomised to the more similar arms |
Effect estimates of inferior arms are less precise as they are stopped early |
Allocation ratio | Randomisation ratio is changed to favour the treatment arms with highest probability of being superior | Ethical advantage: less patients randomised to inferior arms Statistical efficiency: more patients randomised to the more similar arms |
Knowledge of the allocation ratio may lead to ethical dilemma to randomise patients to putative inferior arm |
Study population | Exclude future patient subgroups from randomisation to any or all treatment arms if conclusion for these subgroups is reached | Ethical advantage not to expose patients to experimental treatments if conclusion is reached Efficiency: subgroup can be selected for another intervention domain |
Determination of subgroup effects will, on average, require a larger overall sample size |
Changing the sample size
There are several methods that allow adaptation of the sample size during a study. For instance, through conducting frequent interim analyses in order to continue the trial until a reliable conclusion is reached. If done with a fixed maximum sample size, this allows for early termination for superiority or futility (termed “group-sequential design”). It can also be done without a fixed maximum sample size (termed “adaptive group-sequential design”) in which case recalculation of a maximum sample size during each interim analysis is included. This implies that the trial doesn’t stop as long as the interim result is inconclusive, and thus the planned maximum sample size can increase during the study. Adaptive sample sizes have been rarely applied in the ICU setting (Table 2) whereas they would have been beneficial in many studies in critical care medicine, such as the recent trial comparing hydrocortisone to placebo in sepsis patients [12]. Although the difference in 90-day mortality was not statistically significant, the confidence interval included a relevant effect size (95% CI for the OR 0.82–1.10). In an adaptive design, randomisation could have continued (assuming sufficient funding) until a clinically relevant benefit was convincingly demonstrated or excluded. Arguably, the study would have been more expensive, but also more informative, with research budget better spent.
Table 2.
Study | Population | Intervention | Adaptive rule | Study result |
---|---|---|---|---|
McCloskey et al. [6] | Septic shock with or without GNB | Human monoclonal antibody (HA-1A) vs. placebo | Group sequential design with an interval of 500 GNB patients. Stopping rules: 1) Superiority in patients with GNB, 2) inferiority in patients without GNB. Maximum sample size: 1500 with GNB | Stopped after first interim analysis because of inferiority in patients without GNB (p = 0.09). No benefit for patients with GNB |
Van Nieuwenhoven et al. [7] | Critically ill patients undergoing mechanical ventilation | Semirecumbent position vs. standard care | Group sequential design with an interval of ten patients. Stopping rules: (1) superiority, (2) futility. Maximum sample size: 252 | Stopped after inclusion of 210 patients because of futility |
Zhang et al. [8] | Critically ill patients with septic shock and/or ARDS | PiCCO vs. central venous pressure monitoring | Group sequential design with an interval of 50 patients. Stopping rules: (1) superiority, (2) futility. Maximum sample size: 715 | Stopped after 350 patients because of futility |
Vincent et al. [9] | Patients with severe sepsis | Talactoferrin vs. placebo | Seamless phase II/III design. Decision rule after phase II (n = 350): if results suggest benefit, continue enrolment for (phase III). Planned sample size: 1280 | Stopped after 305 patients for futility and safety concerns |
Welte et al. [10] | Severe community-acquired pneumonia | IGM-enriched immunoglobulin preparation (trimedulin) vs. placebo | Adaptive group sequential design. First interim analysis after 40 patients. Stopping rules: (1) superiority, (2) futility. Adaptation rule: adjust maximum sample size. Original maximum sample size: 82 | During first interim analysis original sample size was increased to 160. At second interim analysis (100 patients) no stopping rule reached. Final analysis was inconclusive |
ARDS acute respiratory distress syndrome, GNB gram-negative bacteraemia, PiCCO pulse contour cardiac output
Changing the intervention
Adaptation can be suitable when comparing more than two different drugs, dosages and/or durations of treatment for the same indication. For instance, in a study of cryptococcal meningitis, three different dosing regimens of liposomal amphotericin B + fluconazole were compared to the standard dosing regimen in the first 160 patients (40 per arm), and only the best faring dosage was compared to standard dosage in the next 300 patients (150 per arm) [13]. This adaptation is referred to as a “drop-the-loser” or “pick-the-winner” design and is often applied in dose-finding studies.
Changing the allocation ratio
Response-adaptive randomisation means that the allocation ratio of randomised interventions is changed during the study based on the results of interim analyses. For instance, consider a three-arm trial with an initial allocation ratio of 1:1:1 for arms A, B, and C. In the first interim analysis, A and B have a better outcome, although C is not statistically significantly inferior. Based on a pre-defined plan, the allocation ratio could be changed to 2:2:1, with less patients being randomised to C. In a subsequent interim analysis C may be found inferior and will then be dropped, leaving more patients for the comparison of A versus B. This was applied in a trial of gepotidacin in three different dosage regimens for patients with acute bacterial skin infections [14]. After the first interim analysis, less patients were randomized to the highest dose regimen, and this arm was dropped at the fourth interim analysis.
Changing the study population
Subgroup-specific effects, e.g. due to differences in pathophysiology, risk of side effects, or pharmacology, occur in many interventions. By measuring subgroup effects during interim analyses, all aforementioned adaptations can be applied to subgroups. An example of this is the I-SPY2 trial on chemotherapy regimens in stage-II/III breast cancer patients with eight biomarker-based subgroups. The investigators recently published the results for one of these subgroups, while in the meantime the trial goes on to determine the optimal treatment for the other subgroups [15].
Advantages of adaptive designs
The adaptive design may have many advantages, most of which are not specific to infectious diseases. Patients have the advantage of a higher chance of receiving better treatment. For researchers and funders there is reasonable chance (though without guarantee) that research questions can be answered with fewer patients, leading to more efficient use of research recourses. Finally, in the case of infectious diseases, adaptive trials may include study domains to be activated in case of emerging diseases or epidemics.
Requirements for adaptive designs
The complexity of the statistical analyses of adaptive trials should not be underestimated. First, there is a need to account for multiple testing due to the frequent interim analyses. Second, due to low numbers within subgroups, imbalance of baseline characteristics is possible, which needs to be corrected for during each interim analysis. Third, time trends may confound effects, particularly if response adaptive randomisation is used. Fourth, as more adaptations are implemented, operational characteristics such as the expected sample size and the chance of incorrect conclusions cannot be calculated with standard approaches, but require simulation studies. Therefore, involvement of qualified statisticians is required, and a detailed statistical analysis plan specifying all possible adaptations must be designed before the study starts.
Conclusion
As compared to the classical RCT, adaptive trials can answer research questions in a more efficient and effective way, but require an extensive and much more complex statistical preparation. Broader use of adaptive trials is expected to improve the cost–benefit ratio of clinical trials in critically ill patients.
Acknowledgements
This work was supported by the Innovative Medicines Initiative Joint Undertaking (IMI JU) (Grant number 115523), Combatting Bacterial Resistance in Europe (COMBACTE), resources of which are composed of financial contribution from the European Union’s 7th Framework Programme (FP7/2007 ± 2013) and the European Federation of Pharmaceutical Industries and Associations (EFPIA) companies’ in kind contribution.
Compliance with ethical standards
Conflicts of interest
The authors declare that they have no conflict of interest related to the topic of this paper.
References
- 1.Dent L, Raftery J. Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme. Trials. 2011;12:109. doi: 10.1186/1745-6215-12-109. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Djulbegovic B, Kumar A, Soares HP, et al. Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Arch Intern Med. 2008;168:632–642. doi: 10.1001/archinte.168.6.632. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Ecarnot F, Quenot J-P, Besch G, Piton G. Ethical challenges involved in obtaining consent for research from patients hospitalized in the intensive care unit. Ann Transl Med. 2017;5:S41. doi: 10.21037/atm.2017.04.42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Berry DA. Adaptive clinical trials in oncology. Nat Rev Clin Oncol. 2012;9:199–207. doi: 10.1038/nrclinonc.2011.165. [DOI] [PubMed] [Google Scholar]
- 5.Simon R. Adaptive treatment assignment methods and clinical trials. Biometrics. 1977;33:743–749. doi: 10.2307/2529473. [DOI] [PubMed] [Google Scholar]
- 6.McCloskey RV, Straube RC, Sanders C, et al. Treatment of septic shock with human monoclonal antibody HA-1A. A randomized, double-blind, placebo-controlled trial. CHESS Trial Study Group. Ann Intern Med. 1994;121:1–5. doi: 10.7326/0003-4819-121-1-199407010-00001. [DOI] [PubMed] [Google Scholar]
- 7.van Nieuwenhoven CA, Vandenbroucke-Grauls C, van Tiel FH, et al. Feasibility and effects of the semirecumbent position to prevent ventilator-associated pneumonia: a randomized study. Crit Care Med. 2006;34:396–402. doi: 10.1097/01.CCM.0000198529.76602.5E. [DOI] [PubMed] [Google Scholar]
- 8.Zhang Z, Ni H, Qian Z. Effectiveness of treatment based on PiCCO parameters in critically ill patients with septic shock and/or acute respiratory distress syndrome: a randomized controlled trial. Intensive Care Med. 2015;41:444–451. doi: 10.1007/s00134-014-3638-4. [DOI] [PubMed] [Google Scholar]
- 9.Vincent J-L, Marshall JC, Dellinger RP, et al. Talactoferrin in severe sepsis. Crit Care Med. 2015;43:1832–1838. doi: 10.1097/CCM.0000000000001090. [DOI] [PubMed] [Google Scholar]
- 10.Welte T, Dellinger RP, Ebelt H, et al. Efficacy and safety of trimodulin, a novel polyclonal antibody preparation, in patients with severe community-acquired pneumonia: a randomized, placebo-controlled, double-blind, multicenter, phase II trial (CIGMA study) Intensive Care Med. 2018;44:438–448. doi: 10.1007/s00134-018-5143-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Bothwell LE, Avorn J, Khan NF, Kesselheim AS. Adaptive design clinical trials: a review of the literature and ClinicalTrials.gov. BMJ Open. 2018;8:e018320. doi: 10.1136/bmjopen-2017-018320. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Venkatesh B, Finfer S, Cohen J, et al. Adjunctive glucocorticoid therapy in patients with septic shock. N Engl J Med. 2018;378:797–808. doi: 10.1056/NEJMoa1705835. [DOI] [PubMed] [Google Scholar]
- 13.Molefi M, Chofle AA, Molloy SF, et al. AMBITION-cm: intermittent high dose Am Bisome on a high dose fluconazole backbone for cryptococcal meningitis induction therapy in sub-Saharan Africa: study protocol for a randomized controlled trial. Trials. 2015;16:276. doi: 10.1186/s13063-015-0799-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.O’Riordan W, Tiffany C, Scangarella-Oman N, et al. Efficacy, safety, and tolerability of gepotidacin (GSK2140944) in the treatment of patients with suspected or confirmed gram-positive acute bacterial skin and skin structure infections. Antimicrob Agents Chemother. 2017;61:e02095-16. doi: 10.1128/AAC.02095-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Rugo HS, Olopade OI, DeMichele A, et al. Adaptive randomization of veliparib-carboplatin treatment in breast cancer. N Engl J Med. 2016;375:23–34. doi: 10.1056/NEJMoa1513749. [DOI] [PMC free article] [PubMed] [Google Scholar]