Platform Trials — Beware the Noncomparable Control Group

To the Editor: The coronavirus disease 2019 (Covid-19) pandemic has highlighted the crucial role of randomized trials in guiding clinical practice and the need for designs that provide rapid evaluation of multiple interventions. Multigroup randomized clinical trials in which multiple experimental treatment groups are compared with a single control group allow for an efficient use of resources in that a separate control group does not need to be generated for each comparison.¹ Platform trials are multigroup trials that permit experimental groups to enter and exit the trial at different times. However, it is important to remember that the unique evidentiary status of randomized trials is based on their ability to guarantee that the trial groups have (on average) equivalent patient populations that allow between-group differences in outcomes to be attributed to the intervention. This generally does not apply to designs that compare trial groups that include patients who did not undergo randomization concurrently.

An example of how a nonconcurrent control group could bias the results of a trial is provided in Figure 1. Consider the decline in in-hospital mortality from Covid-19 that occurred over a 2-month period in the spring of 2020² and a hypothetical trial that compared a control treatment with an ineffective new agent that was not included in the randomization until the second month. If comparisons were made between the patients who received the control treatment during the 2-month period (April–May 2020) and the patients who were randomly assigned to receive the ineffective new agent over the 1-month period (May 2020), the results would erroneously suggest that 30-day mortality would be 37% lower with the ineffective new agent. The same potential for bias occurs when randomization ratios between trial groups are allowed to change over time,³ when some countries (or sites) that have different mortality rates restrict randomization to a specific subset of trial agents, when some countries (or sites) do not allow randomization to the control group,⁴ or when administered concomitant treatments are different between trial treatment groups. Statistical modeling of trends over time and country effects can attempt to ameliorate potential bias due to a noncomparable control group, but there are two weaknesses of this approach. The first is that the more modeling conducted, the less efficient the design is in terms of required sample sizes.³ More importantly, one never knows whether the modeling has successfully eliminated all potential bias.

Figure 1. Hypothetical Example of How Nonconcurrent Randomization Could Bias the Results of a Trial.

Panel A shows the 30-day in-hospital mortality from Covid-19 in April 2020 (12.9% [SE, 0.3]), in May 2020 (5.9% [SE, 0.2]), and over both months (9.3 [SE, 0.2]) (data are from eFig. 2B in Asch et al.²). Panel B shows the data from a hypothetical trial for an ineffective new agent used in May 2020 as compared with a control treatment used in April and May 2020. The data show that mortality was lower by 37% with the ineffective new agent than with the control treatment.

An example of how nonconcurrent randomization (and the other potential aforementioned biases) can complicate interpretation of trial results is the analysis of the efficacy of interleukin-6 blockade with tocilizumab or sarilumab in patients with Covid-19 in the Randomized, Embedded, Multifactorial Adaptive Platform Trial for Community-Acquired Pneumonia (REMAP-CAP). In this international platform trial, the control group used in the analysis was not restricted to patients who had undergone concurrent randomization, and both trial agents were reported to improve survival.⁵ This trial led some treatment guideline committees to recommend the use of these agents (www.cas.mhra.gov.uk/ViewandAcknowledgment/ViewAttachment.aspx?Attachment_id=103745). One cannot say with certainty that the statistical modeling was not successful in eliminating bias, especially in a complex and hard to understand platform trial such as REMAP-CAP. The added value from this trial relative to the other randomized trials with straightforward, comparable controls can be questioned.