Skip to main content
Springer logoLink to Springer
. 2018 Oct 30;45(5):678–682. doi: 10.1007/s00134-018-5426-z

Adaptive designs in clinical trials in critically ill patients: principles, advantages and pitfalls

C H van Werkhoven 1,, S Harbarth 2,3, M J M Bonten 1,4
PMCID: PMC6483961  PMID: 30377740

Introduction

Randomised controlled trials (RCTs) are the gold standard for a comparative evaluation of interventions. Their robust design helps prevent different biases, most importantly confounding by indication. However, RCTs often require large numbers of patients, and even then many appear to be underpowered—and thus inconclusive—due to misspecification of original assumptions used for sample size calculation [1, 2]. Furthermore, especially in critically ill patients, it is difficult to acquire informed consent for interventions that need to start immediately, such as treatment of infections. This may result in selected populations, reducing the generalisability of study findings [3]. Adaptive trials are trials that include decision rules to change key trial design elements during the RCT. The promise of adaptive trials is to provide answers to therapeutic research questions as efficiently as possible without compromising reliability. They can be designed such that a conclusive answer is always reached and that—during the course of the study—the proportion of patients receiving the most promising treatment increases [4]. This benefit for individual patients may overcome ethical barriers to apply deferred or waived consent for randomisation, and thereby increase generalisability of the results. In this viewpoint we aim to elucidate principles, advantages and pitfalls of adaptive trials.

The first adaptive trials were performed in the 1970s, but were not widely adopted due to methodological shortcomings, lack of understanding by clinical investigators, and ethical concerns about weighted randomisation [5]. To the best of our knowledge, in critically ill patients only five adaptive trials have been performed (all using adaptive sample sizes [610]) and one is ongoing (ClinicalTrials.gov NCT02735707). As recent improvements now overcome most of the methodological and technological shortcomings, adaptive designs are gaining more attention [11].

What is an adaptive trial?

Key trial design elements that could be subject to adaptation during the RCT are (1) sample size, (2) intervention arms, (3) allocation ratio, and (4) study population (Table 1). As a result, adaptive trials will—upfront—always have an unknown sample size. Importantly, adaptive trials do not provide a free ticket for trial adaptations: adaptations are based on the analyses of accumulating data with adaptation rules being pre-specified in the study protocol.

Table 1.

Most frequently involved design elements in adaptive trials

Design element Adaptation Advantages Risks and challenges
Sample size Stopping rule if superiority is met. May also include stopping rule for equivalence Efficiency/utility: Sample size determined by true rather than assumed effect size. Study can be designed such that it always reaches a conclusion
Ethical advantage not to expose patients to experimental treatments if conclusion is already reached and not to enrol patients into a trial that has a fair chance not to reach a conclusion
Study may become larger than expected due to small effect size or by chance. Funding agents may be reluctant to support studies with unknown sample size
Interventions Drop inferior intervention arms. May also include rules to add intervention arms Ethical advantage: less patients randomised to inferior arms
Statistical efficiency: more patients randomised to the more similar arms
Effect estimates of inferior arms are less precise as they are stopped early
Allocation ratio Randomisation ratio is changed to favour the treatment arms with highest probability of being superior Ethical advantage: less patients randomised to inferior arms
Statistical efficiency: more patients randomised to the more similar arms
Knowledge of the allocation ratio may lead to ethical dilemma to randomise patients to putative inferior arm
Study population Exclude future patient subgroups from randomisation to any or all treatment arms if conclusion for these subgroups is reached Ethical advantage not to expose patients to experimental treatments if conclusion is reached
Efficiency: subgroup can be selected for another intervention domain
Determination of subgroup effects will, on average, require a larger overall sample size

Changing the sample size

There are several methods that allow adaptation of the sample size during a study. For instance, through conducting frequent interim analyses in order to continue the trial until a reliable conclusion is reached. If done with a fixed maximum sample size, this allows for early termination for superiority or futility (termed “group-sequential design”). It can also be done without a fixed maximum sample size (termed “adaptive group-sequential design”) in which case recalculation of a maximum sample size during each interim analysis is included. This implies that the trial doesn’t stop as long as the interim result is inconclusive, and thus the planned maximum sample size can increase during the study. Adaptive sample sizes have been rarely applied in the ICU setting (Table 2) whereas they would have been beneficial in many studies in critical care medicine, such as the recent trial comparing hydrocortisone to placebo in sepsis patients [12]. Although the difference in 90-day mortality was not statistically significant, the confidence interval included a relevant effect size (95% CI for the OR 0.82–1.10). In an adaptive design, randomisation could have continued (assuming sufficient funding) until a clinically relevant benefit was convincingly demonstrated or excluded. Arguably, the study would have been more expensive, but also more informative, with research budget better spent.

Table 2.

Examples of adaptive trials in critically ill patients, all using adaptive sample size only

Study Population Intervention Adaptive rule Study result
McCloskey et al. [6] Septic shock with or without GNB Human monoclonal antibody (HA-1A) vs. placebo Group sequential design with an interval of 500 GNB patients. Stopping rules: 1) Superiority in patients with GNB, 2) inferiority in patients without GNB. Maximum sample size: 1500 with GNB Stopped after first interim analysis because of inferiority in patients without GNB (p = 0.09). No benefit for patients with GNB
Van Nieuwenhoven et al. [7] Critically ill patients undergoing mechanical ventilation Semirecumbent position vs. standard care Group sequential design with an interval of ten patients. Stopping rules: (1) superiority, (2) futility. Maximum sample size: 252 Stopped after inclusion of 210 patients because of futility
Zhang et al. [8] Critically ill patients with septic shock and/or ARDS PiCCO vs. central venous pressure monitoring Group sequential design with an interval of 50 patients. Stopping rules: (1) superiority, (2) futility. Maximum sample size: 715 Stopped after 350 patients because of futility
Vincent et al. [9] Patients with severe sepsis Talactoferrin vs. placebo Seamless phase II/III design. Decision rule after phase II (n = 350): if results suggest benefit, continue enrolment for (phase III). Planned sample size: 1280 Stopped after 305 patients for futility and safety concerns
Welte et al. [10] Severe community-acquired pneumonia IGM-enriched immunoglobulin preparation (trimedulin) vs. placebo Adaptive group sequential design. First interim analysis after 40 patients. Stopping rules: (1) superiority, (2) futility. Adaptation rule: adjust maximum sample size. Original maximum sample size: 82 During first interim analysis original sample size was increased to 160. At second interim analysis (100 patients) no stopping rule reached. Final analysis was inconclusive

ARDS acute respiratory distress syndrome, GNB gram-negative bacteraemia, PiCCO pulse contour cardiac output

Changing the intervention

Adaptation can be suitable when comparing more than two different drugs, dosages and/or durations of treatment for the same indication. For instance, in a study of cryptococcal meningitis, three different dosing regimens of liposomal amphotericin B + fluconazole were compared to the standard dosing regimen in the first 160 patients (40 per arm), and only the best faring dosage was compared to standard dosage in the next 300 patients (150 per arm) [13]. This adaptation is referred to as a “drop-the-loser” or “pick-the-winner” design and is often applied in dose-finding studies.

Changing the allocation ratio

Response-adaptive randomisation means that the allocation ratio of randomised interventions is changed during the study based on the results of interim analyses. For instance, consider a three-arm trial with an initial allocation ratio of 1:1:1 for arms A, B, and C. In the first interim analysis, A and B have a better outcome, although C is not statistically significantly inferior. Based on a pre-defined plan, the allocation ratio could be changed to 2:2:1, with less patients being randomised to C. In a subsequent interim analysis C may be found inferior and will then be dropped, leaving more patients for the comparison of A versus B. This was applied in a trial of gepotidacin in three different dosage regimens for patients with acute bacterial skin infections [14]. After the first interim analysis, less patients were randomized to the highest dose regimen, and this arm was dropped at the fourth interim analysis.

Changing the study population

Subgroup-specific effects, e.g. due to differences in pathophysiology, risk of side effects, or pharmacology, occur in many interventions. By measuring subgroup effects during interim analyses, all aforementioned adaptations can be applied to subgroups. An example of this is the I-SPY2 trial on chemotherapy regimens in stage-II/III breast cancer patients with eight biomarker-based subgroups. The investigators recently published the results for one of these subgroups, while in the meantime the trial goes on to determine the optimal treatment for the other subgroups [15].

Advantages of adaptive designs

The adaptive design may have many advantages, most of which are not specific to infectious diseases. Patients have the advantage of a higher chance of receiving better treatment. For researchers and funders there is reasonable chance (though without guarantee) that research questions can be answered with fewer patients, leading to more efficient use of research recourses. Finally, in the case of infectious diseases, adaptive trials may include study domains to be activated in case of emerging diseases or epidemics.

Requirements for adaptive designs

The complexity of the statistical analyses of adaptive trials should not be underestimated. First, there is a need to account for multiple testing due to the frequent interim analyses. Second, due to low numbers within subgroups, imbalance of baseline characteristics is possible, which needs to be corrected for during each interim analysis. Third, time trends may confound effects, particularly if response adaptive randomisation is used. Fourth, as more adaptations are implemented, operational characteristics such as the expected sample size and the chance of incorrect conclusions cannot be calculated with standard approaches, but require simulation studies. Therefore, involvement of qualified statisticians is required, and a detailed statistical analysis plan specifying all possible adaptations must be designed before the study starts.

Conclusion

As compared to the classical RCT, adaptive trials can answer research questions in a more efficient and effective way, but require an extensive and much more complex statistical preparation. Broader use of adaptive trials is expected to improve the cost–benefit ratio of clinical trials in critically ill patients.

Acknowledgements

This work was supported by the Innovative Medicines Initiative Joint Undertaking (IMI JU) (Grant number 115523), Combatting Bacterial Resistance in Europe (COMBACTE), resources of which are composed of financial contribution from the European Union’s 7th Framework Programme (FP7/2007 ± 2013) and the European Federation of Pharmaceutical Industries and Associations (EFPIA) companies’ in kind contribution.

Compliance with ethical standards

Conflicts of interest

The authors declare that they have no conflict of interest related to the topic of this paper.

References

  • 1.Dent L, Raftery J. Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme. Trials. 2011;12:109. doi: 10.1186/1745-6215-12-109. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Djulbegovic B, Kumar A, Soares HP, et al. Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Arch Intern Med. 2008;168:632–642. doi: 10.1001/archinte.168.6.632. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Ecarnot F, Quenot J-P, Besch G, Piton G. Ethical challenges involved in obtaining consent for research from patients hospitalized in the intensive care unit. Ann Transl Med. 2017;5:S41. doi: 10.21037/atm.2017.04.42. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Berry DA. Adaptive clinical trials in oncology. Nat Rev Clin Oncol. 2012;9:199–207. doi: 10.1038/nrclinonc.2011.165. [DOI] [PubMed] [Google Scholar]
  • 5.Simon R. Adaptive treatment assignment methods and clinical trials. Biometrics. 1977;33:743–749. doi: 10.2307/2529473. [DOI] [PubMed] [Google Scholar]
  • 6.McCloskey RV, Straube RC, Sanders C, et al. Treatment of septic shock with human monoclonal antibody HA-1A. A randomized, double-blind, placebo-controlled trial. CHESS Trial Study Group. Ann Intern Med. 1994;121:1–5. doi: 10.7326/0003-4819-121-1-199407010-00001. [DOI] [PubMed] [Google Scholar]
  • 7.van Nieuwenhoven CA, Vandenbroucke-Grauls C, van Tiel FH, et al. Feasibility and effects of the semirecumbent position to prevent ventilator-associated pneumonia: a randomized study. Crit Care Med. 2006;34:396–402. doi: 10.1097/01.CCM.0000198529.76602.5E. [DOI] [PubMed] [Google Scholar]
  • 8.Zhang Z, Ni H, Qian Z. Effectiveness of treatment based on PiCCO parameters in critically ill patients with septic shock and/or acute respiratory distress syndrome: a randomized controlled trial. Intensive Care Med. 2015;41:444–451. doi: 10.1007/s00134-014-3638-4. [DOI] [PubMed] [Google Scholar]
  • 9.Vincent J-L, Marshall JC, Dellinger RP, et al. Talactoferrin in severe sepsis. Crit Care Med. 2015;43:1832–1838. doi: 10.1097/CCM.0000000000001090. [DOI] [PubMed] [Google Scholar]
  • 10.Welte T, Dellinger RP, Ebelt H, et al. Efficacy and safety of trimodulin, a novel polyclonal antibody preparation, in patients with severe community-acquired pneumonia: a randomized, placebo-controlled, double-blind, multicenter, phase II trial (CIGMA study) Intensive Care Med. 2018;44:438–448. doi: 10.1007/s00134-018-5143-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Bothwell LE, Avorn J, Khan NF, Kesselheim AS. Adaptive design clinical trials: a review of the literature and ClinicalTrials.gov. BMJ Open. 2018;8:e018320. doi: 10.1136/bmjopen-2017-018320. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Venkatesh B, Finfer S, Cohen J, et al. Adjunctive glucocorticoid therapy in patients with septic shock. N Engl J Med. 2018;378:797–808. doi: 10.1056/NEJMoa1705835. [DOI] [PubMed] [Google Scholar]
  • 13.Molefi M, Chofle AA, Molloy SF, et al. AMBITION-cm: intermittent high dose Am Bisome on a high dose fluconazole backbone for cryptococcal meningitis induction therapy in sub-Saharan Africa: study protocol for a randomized controlled trial. Trials. 2015;16:276. doi: 10.1186/s13063-015-0799-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.O’Riordan W, Tiffany C, Scangarella-Oman N, et al. Efficacy, safety, and tolerability of gepotidacin (GSK2140944) in the treatment of patients with suspected or confirmed gram-positive acute bacterial skin and skin structure infections. Antimicrob Agents Chemother. 2017;61:e02095-16. doi: 10.1128/AAC.02095-16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Rugo HS, Olopade OI, DeMichele A, et al. Adaptive randomization of veliparib-carboplatin treatment in breast cancer. N Engl J Med. 2016;375:23–34. doi: 10.1056/NEJMoa1513749. [DOI] [PMC free article] [PubMed] [Google Scholar]

Articles from Intensive Care Medicine are provided here courtesy of Springer

RESOURCES