Pragmatic trials with prespecified subgroups: what oncologists can learn from COVID-19

Abstract

Randomized controlled trials designed to test cancer therapies often fail to clarify effectiveness in real-world settings. Herein, we explore lessons for trial development in oncology that can be learnt from the large-cohort, pragmatic RECOVERY trial involving patients hospitalized with COVID-19.

Refers to RECOVERY Collaborative Group. Dexamethasone in hospitalized patients with covid-19 - preliminary report. N. Engl. J. Med. 10.1056/NEJMoa2021436 (2020).

Participants in randomized controlled trials (RCTs) of anticancer therapies are often younger and fitter than the average patient with their specific malignancy. This discrepancy might explain the efficacy–effectiveness gap, whereby the efficacy of cancer therapies in RCTs often exceeds their corresponding effectiveness in clinical practice¹. The outcomes of patients with cancer receiving treatment in RCTs do not mirror those of the general population for several reasons: these include the use of restrictive inclusion criteria, registrational logistics favouring enrolment at academic medical centres and referral bias favouring enrolment of healthier patients with more indolent disease. By contrast, the multi-arm randomized RECOVERY trial involving patients hospitalized with coronavirus disease 2019 (COVID-19) enrolled over 11,000 ‘real-world’ patients from 173 different centres in the UK, comprising 15% of the UK’s eligible population². Recently published results from this study demonstrate that the addition of dexamethasone to usual care leads to a reduction in 28-day mortality (age-adjusted rate ratio (RR) 0.83, 95% CI 0.75–0.93; P < 0.001), owing to reductions in mortality among patients receiving invasive mechanical ventilation or supplementary oxygen only, but not among patients not requiring supplementary oxygen. Beyond these practice-changing results, the RECOVERY study offers two broader lessons for clinical trials in oncology: 1) the merits of inclusivity to maintain generalizability, and 2) the potential to recruit broader populations while preserving core efficacy subgroups. We explore these two lessons here.

…the RECOVERY study offers two broader lessons for clinical trials in oncology: 1) the merits of inclusivity to maintain generalizability, and 2) the potential to recruit broader populations while preserving core efficacy subgroups

First, the RECOVERY trial enrolled UK residents as they are, not just young and relatively healthy patients who happened to present to an academic centre. Barriers to enrolment were intentionally minimal: the protocol had no exclusion criteria based on age or organ function, and the web eligibility-determination form did not query baseline laboratory values in its first iteration. The benefits of these simple eligibility criteria are apparent in the demographics of patients enrolled in the RECOVERY trial: 27% of patients had heart disease, while almost 10% had either kidney or liver disease. Patients ≥80 years of age comprised almost a quarter of both treatment arms, which is particularly relevant given the propensity of COVID-19 to affect older patients more severely³. Even patients requiring mechanical ventilation owing to disease-related hypoxia were eligible for enrolment. By contrast, patients with cancer showing signs of disease-related deconditioning (manifesting as inferior performance status) are routinely excluded from clinical trials involving oncological interventions. Similarly, older patients with cancer are profoundly underrepresented in clinical trials designed to test anticancer drugs⁴. The RECOVERY trial’s pragmatic design⁵, both with regard to eligibility criteria and follow-up monitoring, provides a model for clinical trials in oncology and particularly for the later phases of drug development.

Another noteworthy feature of the RECOVERY trial was the ease of patient registration. Data entry and randomization for the RECOVERY trial were completed within minutes using a simple online form with pre-randomization prompts to ensure allocation only to treatments that were immediately available in-house. While other components of patients’ medical records could be (and will be) queried for later more granular analyses, such components were minimized at the point of enrolment onto RECOVERY. By contrast, the process of recruiting and randomizing a single patient with cancer to a phase III oncology trial might take 10–20 hours of collective effort by study investigators and other research team members⁶. Given the budgetary and other constraints facing researchers globally owing to the COVID-19 pandemic, studies that minimize non-essential data entry and rapidly randomize patients only to readily available treatments are both practical for investigators and appealing for patients. Such pragmatically designed studies might also increase the representativeness of oncology research through increased enrolment of patients from community-based practices, which is an important priority for our field and our patients.

Two main drawbacks exist to pragmatic RCTs. Firstly, given the ease of enrolment in the RECOVERY trial, we admittedly know little about the 15% of patients deemed unsuitable for randomization to dexamethasone. Secondly, pragmatic trials come with a risk of failing to identify a benefit in a core ‘ideal’ population owing to a lack of benefit in a broader ‘average’ population. While admittedly a risk, the RECOVERY trial elegantly addressed this concern through the use of pre-specified interaction tests to discern differences in mortality based on receipt of baseline respiratory support, age group, gender and symptom duration. As such, it was possible for RECOVERY to have detected a benefit only in younger, healthier patients even if the overall outcome was negative. Ultimately, however, a reduction in short-term mortality associated with dexamethasone was shown among the subpopulations of patients receiving supplementary oxygen or invasive mechanical ventilation; the corresponding RRs for 28-day mortality using regression models employing interaction tests were 0.82 (95% CI 0.72–0.94) and 0.64 (95% CI 0.51–0.81), respectively². By contrast, patients who did not receive supplementary oxygen did not derive a mortality benefit from dexamethasone (RR 1.19, 95% CI 0.91–1.55). As such, the overall benefit from dexamethasone for hospitalized patients with COVID-19 was entirely driven by the inclusion of patients from subgroups with the most severe symptoms and not from those in the healthier subgroups.

Compared with patients who did not receive supplementary oxygen, for whom the anti-inflammatory effects of dexamethasone might be unnecessary or even unhelpful, patients requiring supplementary oxygen as a minimum are thus the core efficacy group for dexamethasone as a treatment for COVID-19. The trial’s investigators might have suspected but could not possibly guarantee this observation before launching their study in the midst of a global pandemic. However, their use of a priori interaction testing enabled them to randomize and rigorously investigate both relatively healthy and relatively ill patients without a reliance on single-arm expansion cohorts or expanded access programmes paired against historical controls. Is this paradigm feasible within oncology as well? Therapies might offer different risk:benefit profiles for patients who do not meet traditional eligibility criteria for RCTs, which is a compelling indication for their inclusion in these studies⁷. The historical rationale for excluding older patients and/or those with lower performance status from clinical trials is their often lower absolute survival; however, the findings of a meta-analysis of data from 66 RCTs involving 44,511 patients indicate that advanced age and performance status do not affect relative magnitudes of benefit from modern cancer therapies⁸. Yet, little is known about patients who were not enrolled in these studies, who may be older and more frail. In RECOVERY, even if patients receiving supplementary oxygen and/or mechanical ventilation had been expected to have higher short-term mortality rates than those who did not receive supplementary oxygen, their randomization enables the investigation of these specific subgroups while also preserving the ability to detect benefits in healthier subgroups.

The approach taken by the RECOVERY investigators leads to another possible risk associated with pragmatic studies: what if the inclusion of real-world patients into an RCT dilutes the number of responses to the extent that the outcome of the entire trial is negative? This possibility underpins the efficacy–effectiveness gap in oncology. For example, if the registry studies testing the efficacy of sorafenib or regorafenib for hepatocellular carcinoma had enrolled patients with worsened cirrhosis or performance status, these studies might have revealed only marginal benefits, and their regulatory approvals might have been thwarted^7,9. However, the number of real-world patients subjected to an ineffective and, in many cases, harmful intervention would undoubtedly be lower than the number required for this lesson to be realized after regulatory approval. Furthermore, study sponsors could design RCTs to feature core efficacy subgroups in order to highlight populations in which a superior response would be expected. With any given pragmatic RCT, the alluring possibility of a new anticancer therapy demonstrating a relative benefit in sicker as well as healthier patients would also exist. Given the dearth of evidence supporting the use of most anticancer drugs in patients with substantial comorbidities, any studies showing a benefit in these populations would satisfy an unmet need.

…the RECOVERY trial exemplifies many of the qualities of trials that are needed to resolve the efficacy–effectiveness gap in oncology

In conclusion, the RECOVERY trial exemplifies many of the qualities of trials that are needed to resolve the efficacy–effectiveness gap in oncology. In a single pragmatic study, we can learn whether anticancer drugs work in both ideal and real-world patients. A negative finding in one cohort does not jeopardize a potentially positive finding for the other, and — no matter what is discovered — the results of such an RCT will enable more precise administration of cancer therapies in the real world. The COVID-19 pandemic is already teaching us the advantages of easing ongoing data requirements for current RCTs¹⁰. The RECOVERY trial goes a step further by showing us how future RCTs in oncology should look: easier for frontline oncologists to operationalize; easier for real-world patients to join; and easier for the public to extrapolate results from.

Competing interests

V.P. has acted as a consultant of UnitedHealthcare, has received research funding from Arnold Ventures, has received royalties from Johns Hopkins Press and Medscape, has received speaker’s fees from Evicore and for Grand Rounds/lectures from various universities, medical centres, non-profit organizations, and professional societies and has received financial support for the Plenary Session series of podcasts via Patreon. R.B. declares no competing interests.