Analysis of Control Arm Quality in Randomized Clinical Trials Leading to Anticancer Drug Approval by the US Food and Drug Administration

Key Points

Question

How often are anticancer drugs approved by the US Food and Drug Administration (FDA) based on randomized clinical trials designed with a suboptimal control arm?

Findings

In this quality improvement study, 143 anticancer drug approvals granted by the FDA between January 1, 2013, and July 31, 2018, were reviewed; 16 (17%) of 96 anticancer drugs approved for market were approved based on comparing them with suboptimal control arms.

Meaning

Despite the increase in the number of drug approvals by the FDA, a substantial number of anticancer drugs are receiving market authorizations based on data that does not prove their superiority over standard-of-care therapy, thereby leaving both clinicians and patients unsure of the benefit conveyed by these agents.

This quality improvement study analyzed the quality of control arms in pivotal randomized clinical trials of anticancer drugs that led to approval by the US Food and Drug Administration between January 1, 2013, and July 31, 2018.

Abstract

Importance

To date, an empirical evaluation of the quality of control arms in randomized clinical trials (RCTs) leading to anticancer drug approvals by the US Food and Drug Administration (FDA) has not been undertaken.

Objective

We sought to estimate the percentage of RCTs that used a control arm deemed suboptimal and led to FDA approval of anticancer drugs from January 1, 2013, to July 31, 2018.

Design, Setting, and Participants

This quality improvement study included 143 anticancer drug approvals granted by the FDA from January 1, 2013, to July 31, 2018. All approvals based on single-arm studies (47 approvals) were excluded. Approvals based on RCTs were further investigated and each trial was analyzed for design, time of patient accrual, control arm, and primary end point. Standard-of-care therapy was determined by evaluating the literature and published guidelines 1 year prior to the start of trial enrollment. The percentage of approvals based on RCTs that used suboptimal control arms was then calculated. The quality of the control arm was deemed suboptimal if the choice of control agent was restricted to exclude a recommended agent, the control arm was specified but the recommended agent was unspecified, and if prior RCT data had demonstrated that the control agent was inferior to an available alternative.

Main Outcomes and Measures

Estimated percentage of RCTs that used suboptimal control arms that led to FDA approval of anticancer agents between January 1, 2013, to July 31, 2018.

Results

A total of 145 studies that led to 143 drug approvals between January 1, 2013, and July 31, 2018, were included. Of these studies, 47 single-arm studies were excluded. The remaining 98 studies led to 96 drug approvals. Of these 96 approvals, 16 (17%) were based on RCTs with suboptimal control arms; 15 were international trials, and 1 was conducted in the United States. The type of approval was regular in 14 trials and accelerated in 2 trials. When categorized by the nature of suboptimal control, 4 (25%) trials omitted active treatment in control arm by limiting investigator’s choice, 10 (63%) trials omitted active treatment in the control arm by using a control agent known to be inferior to other available agents or not allowing combinations, and 2 (13%) trials used a previously used treatment in the control arm with a known lack of benefit associated with reexposure.

Conclusions and Relevance

Although anticancer drug approvals are increasing, a proportion of these drugs are reaching the market without proven superiority to what is considered the standard of care at the time of patient enrollment in pivotal trials. The choice of control arm should be optimized to ensure that new anticancer agents being marketed are truly superior to what most clinicians would prescribe outside a clinical trial setting.

Introduction

Most randomized clinical trials (RCTs) of novel anticancer drugs leading to marketing authorization by the US Food and Drug Administration (FDA) are designed, conducted, and funded by the biopharmaceutical industry. One potential concern with these studies is that in an effort to increase the likelihood of obtaining favorable results, the control arm of these studies may be suboptimal (“straw man” comparators).^1,2,3,4 For instance, Sharman et al⁵ note that the RESONATE-2 trial, a randomized trial of ibrutinib vs chlorambucil in CLL, relied on chlorambucil as control agent, a drug that has been repeatedly beaten by alternatives and has poor real-world use. Tao and Prasad¹ note other examples of trials with suboptimal control arms, including an RCT comparing nivolumab with dacarbazine in metastatic melanoma that enrolled patients after ipilimumab had been demonstrated to be superior in this setting.

Using a suboptimal control arm may bias a trial in favor of the experimental arm and reduces a trial’s external validity—it is no longer capable of answering the pertinent clinical question of whether the novel drug is superior to prevailing practice. Despite concern for suboptimal control arms in oncology, to our knowledge, there is no empirical analysis assessing how often they occur. For these reasons, we analyzed the percentage of RCTs that used a suboptimal control arm and led to regulatory approval for anticancer drugs.

Methods

This study of published reports did not involve patient data and was not submitted for institutional review board approval. Written patient informed consent was not required. We sought to assess what percentage of RCTs use a suboptimal control arm and lead to new or supplemental marketing authorization of anticancer drugs by the FDA.

Data Set

We examined all hematology/oncology drug approvals by the FDA from January 1, 2013, through July 31, 2018, listed on the FDA website. We identified the accrual period, sites of accrual (national vs international), control arm used, primary end point, date of approval, and pathway of approval (accelerated vs regular). We excluded all marketing authorizations made on the basis of nonrandomized data.

Assessing the Control Arm

We considered the quality of the control arm to be suboptimal if (1) restrictions were placed on the choice of control that excluded a recommended agent, (2) the control arm was specified but not the recommended agent (eg, the control arm was a single agent when doublet therapy is recommended), and (3) prior RCT data had demonstrated the control agent was inferior to an available alternative.

We assessed alternative control arms using 2 independent methods. First, the first and second authors (T.H. and M.B.S.) performed a search of the National Comprehensive Cancer Network (NCCN) guidelines and published review articles for a particular cancer to determine the standard-of-care (SOC) therapy for a specific cancer. These articles were published at least 1 year prior to start of accrual of an RCT of interest that led to FDA approval. Second, the first and second authors separately and independently read the published RCT data as well as the appendices, when relevant, and determined the adequacy of control arm. Conflicts were resolved by referring to the third author (V.P.).

Statistical Analysis

Descriptive statistics are reported throughout. The study data was analyzed from October 1, 2018, to October 15, 2018.

Results

Between January 1, 2013, and July 31, 2018, the FDA granted 143 anticancer drug marketing authorizations. Of these authorizations, 47 were excluded because the pivotal trial was nonrandomized, and 96 anticancer drugs were approved on the basis of 98 RCTs (Figure 1). Of the 96 approvals based on 98 RCTs, 11 (11%) were for genitourinary malignant neoplasms, 8 (8%) for gynecologic malignant neoplasms, 11 (11%) for gastrointestinal and hepatobiliary malignant neoplasms, 17 (18%) for lung and head and neck malignant neoplasms, 15 (16%) were for melanoma and sarcoma, 11 (11%) were for breast neoplasms, 5 (5%) were for myeloid malignant diseases, 12 (13%) were for lymphoid cancers, 6 (6%) were for plasma cell neoplasms (Figure 2 and eTable 1 in the Supplement). Of the 2 approvals for drugs to treat multiple myeloma (daratumumab and lenalidomide), which were based on 2 separate RCTs each,^6,7,8,9 all 4 trials used optimal controls.

Figure 1. Flowchart of All Cancer Drug Approvals From January 1, 2013, to July 31, 2018.

Figure 2. Proportion of Trials That Used a Suboptimal Control Arm by Tumor Type.

Of these 96 anticancer drug approvals, 16 (17%) were based on trials with suboptimal controls^{10,11,12,13,14,15,17,20,21,22,24,25,26,28,31,32} (Table). Of these 16 approvals, 15 were based on international trials,^{10,11,12,13,14,15,17,20,22,24,25,26,28,31,32} and 1 was based on a trial²¹ conducted in the United States. The FDA approval pathway was regular in 14 trials,^{10,11,12,14,15,17,21,22,24,25,26,28,31,32} and accelerated in 2 trials.^13,20 The primary end point for these studies was progression-free survival in 11 (69%),^{10,12,15,17,21,22,24,25,31,32} overall response rate in 2 (13%),^14,20 another surrogate end point (eg, metastasis-free survival, major molecular response) in 2 (13%),^11,13 and overall survival in 1 (6%).^26,28

Table. Randomized Clinical Trials With a Suboptimal Control Arm.

Study	Trial (Accrual Period)	Indication	Investigational Agent	Control	Primary End Point	Comments on Suboptimal Control
Seymour et al,10 2018	MURANO (March 2014-September 2015)	CLL with or without 17p deletion, who have received at least 1 line of therapy	Venetoclax	Bendamustine + rituximab	PFS	Most patients (95%) previously treated with alkylating agent, 78% previously treated with anti-CD20 antibody; therefore many clinicians may not have chosen reexposure to control arm.
Smith et al,11 2018	SPARTAN (October 2013-December 2016)	Nonmetastatic castration–resistant prostate cancer	Apalutamide	Placebo	MFS	Bicalutamide, an AR antagonist, had been used in this setting for several years prior to enrollment on this trial. Patients had to stop bicalutamide therapy before randomization.
Robson et al,12 2017	OlympiAD (April 2014 and November 2015)	Germline BRCA-mutated ERBB2 (HER2)-negative metastatic breast cancer, previously treated with chemo	Olaparib	Investigator’s choice of chemotherapy (capecitabine, eribulin, or vinorelbine)	PFS	Platinum not allowed as investigator’s choice. Efficacy of platinum chemotherapy well documented in BRCA-mutated cancers.
Cortes et al,13 2018	BFORE (July 2014 and August 2015)	First-line chronic phase CML	Bosutinib	Imatinib	MMR at 12 months	Nilotinib and dasatinib had already demonstrated better MMR compared with imatinib.
Prince et al,14 2017	ALCANZA (August 2012 and July 2015)	Primary cutaneous ALCL, or CD30+ expressing mycosis fungoides who received previous therapy	Brentuximab vedotin	Investigator’s choice (oral methotrexate or bexarotene)	ORR lasting 4 months	HDAC inhibitors (vorinostat and romidepsin) not allowed in control despite FDA approval.
Soria et al,15 2017	ASCEND-4 (August 2013 and May 2015)	Metastatic ALK-positive NSCLC	Ceritinib	Platinum-based therapy	PFS	Crizotinib FDA approved in 2011 for ALK-positive NSCLC,16 and was shown to be superior to platinum-based chemotherapy.
Bellmunt et al,17 2017	Keynote-045 (November 2014 and November 2015)	Metastatic urothelial carcinoma who progress on platinum therapy	Pembrolizumab	Investigator choice (paclitaxel, docetaxel, or vinflunine)	PFS	Phase 2 data on nab-paclitaxel or pemetrexed published in 200618,19 but neither allowed in control arm.
Kim et al,20 2017	ALTA (June 2014 and September 2015)	Metastatic ALK-positive NSCLC after crizotinib	Brigatinib	Crizotinib (high-dose)	ORR	Platinum-based doublet therapy SOC after progression on crizotinib, not increasing the dose. Using same drug at higher dose for control arm is suboptimal and unproven.
Tap et al,21 2016	Tap et al. (October 2010 and January 2013)	Soft tissue sarcoma for which anthracycline is appropriate	Olaratumab	Doxorubicin	PFS	Doublet chemo is SOC (docetaxel/gemcitabine, doxorubicin/ifosfamide).
Choueiri et al,22 2016	METEOR (August 2013 through November 2014)	Advanced RCC in following 1 line of therapy	Cabozantinib	Everolimus	PFS	Selected the second-line drug that had not beaten an active competitor. Axitinib for second-line treatment (active against sorafenib) approved in 2012.23 Everolimus compared with placebo.
Yao et al,24 2016	RADIANT-4 (April 2012 and August 2013)	NET, GI, lung	Everolimus	Placebo + BSC	PFS	Standard of care is SSA. Patients randomized to placebo and were not able to receive SSA (about 45% of patients in each arm SSA naïve) with placebo, crossover not allowed except for emergent carcinoid symptoms.
Sehn et al,25 2016	GADOLIN (April 2010 and Sept 2014)	Follicular lymphoma who relapsed after/refractory to rituximab	Obinutuzumab (+ bendamustine)	Bendamustine	PFS	Single-agent benadamustine not optimal, 20% of enrolled patients deemed rituximab refractory after exposure to single agent rituximab without chemotherapy, 40% of enrolled patients deemed refractory while on maintenance rituximab, rituximab active when combined with chemotherapy in both populations.
Moreau et al,26 2016	TOURMALINE-MM1 (August 2012, to May 2014)	Myeloma after at least one prior therapy	Ixazomib (+ lenalidomide + dexamethasone)	Placebo (+ lenalidomide + dexamethasone)	PFS	Triplet regimen salvage therapy standard for fit patients in 2011 (bortezomib /lenalidomide /dexamethasone, CyBorD).27 Moreover, 31% of patients had not received bortezomib, parenteral proteasome inhibitor (ixazomib oral proteasome inhibitor).
Fuchs et al,28 2014	REGARD (October 2009 and January 2012)	Advanced gastric or GEJ adenocarcinoma with disease progression on fluoropyrimidine or platinum-containing therapy	Ramucirumab	Placebo	OS	Taxanes previously shown to be active.29,30 Not allowed to be used in control arm.
Goede et al,31 2014	CLL11 (April 2010 and July 2012)	Previously untreated CLL	Obinutuzumab (+ chlorambucil)	Chlorambucil OR chlorambucil + rituximab	PFS	Almost no US patients. Rituximab optional choice in control arm but should not have been when comparing with another CD-20 antibody.
Yang et al,32 2015	LUX-Lung 3 (August 2009 and February 2011)	First-line metastatic NSCLC with EGFR exon 19 deletions or exon 21 L858R mutations	Afatinib	Cisplatin/pemetrexed	PFS	Gefitinib approved in EU before start of enrollment on trial. Data from Mok et al33 showed superiority of gefinitib to chemotherapy in this population.

Abbreviations: ALCL, anaplastic large cell lymphoma; ALK, anaplastic lymphoma kinase; AR, androgen receptor; CLL, chronic lymphocytic leukemia; CML, chronic myeloid leukemia; CyBorD, cyclophosphamide, bortezomib, and dexamethasone; EGFR, epidermal growth factor receptor; EU, European Union; FDA, Food and Drug Administration; GEJ, gastroesophageal junction; GI, gastrointestinal; HDAC, histone deacetylase; MFS, metastasis-free survival; MMR, major molecular response; NET, neuroendocrine tumors; NSCLC, non–small cell lung cancer; ORR, overall response rate; OS, overall survival; PFS, progression-free survival; RCC, renal cell carcinoma; SOC, standard of care; SSA, somatostatin analogs.

The Table provides a list of all cancer drug approvals determined to have suboptimal control arms, and eTable 2 in the Supplement outlines the reasons they were deemed suboptimal. When categorized by the nature of suboptimal control, 4 (25%) of 16 trials omitted active treatment in control arm by limiting investigator’s choice,^12,14,17,25 10 (63%) of 16 trials omitted active treatment in control arm by using control known to be inferior to other available agents or not allowing combinations,^{11,13,15,21,22,24,26,28,31,32} and 2 (13%) of 16 used a previously used treatment in the control arm with known lack of benefit associated with reexposure^10,20 (eTable 2 in the Supplement).

When the threshold for determining the SOC therapy was extended to 2 years prior to start of accrual of an RCT of interest, 14 of the 16 clinical trials^{10,11,12,13,14,15,17,20,21,22,24,25,28,31} had control arms considered suboptimal.

Discussion

During our study period, 16 of 96 anticancer drugs (17%) gained FDA approval based on RCTs that used a suboptimal control arm for the SOC in the United States. These findings raise important observations. First, international RCTs conducted in other countries may use a suboptimal control arm owing to lack of availability of what would be considered standard therapy in the United States. For example, the ALCANZA study¹⁴ allowed use of oral methotrexate or bexarotene (investigator’s choice) but prohibited the use of histone deacetylase (HDAC) inhibitors, which were approved by the FDA at the time of trial enrollment. This was justified by the lack of approval of HDAC inhibitors in the EU for cutaneous anaplastic large cell lymphoma. Yet, the results of this trial¹⁴ were also submitted for FDA approval. Apart from the ALCANZA trial, international RCT accrual did not directly explain the choice of control arm in 13 out of 16 trials.^{10,11,12,13,14,15,17,20,21,22,24,25,26,28}

Second, most approvals based on RCTs with suboptimal control arms receive regular or full FDA approval, and thereby do not require additional RCTs to verify clinical benefit. This is problematic because when an experimental agent has not been proven to be superior to the established SOC treatment, clinicians are potentially offering patients an agent that may be the equivalent or even inferior to established standard therapy, usually at a higher cost and alternate toxicity profile. The relevant question for the practicing physician, that is, whether a new drug is better than the current best therapy, may not be answered.

Third, the frequency of having a suboptimal control arm was similar between RCTs for hematologic and solid tumors (6 of 25 [24%] vs 10 of 73 [14%]). This lack of difference may be surprising because it appears that options for SOC therapy, particularly for hematologic cancers, can be numerous, allowing for laxity in the choice of control when designing RCTs. For example, TOURMALINE-MM1 compared the addition of ixazomib vs placebo to lenalidomide/dexamethasone in relapsed and/or refractory multiple myeloma.²⁶ Despite some guidelines listing doublet therapy regimens for relapsed multiple myeloma,²⁷ many experts used a triplet regimen³⁴ for relapsed multiple myeloma in the real world setting before the TROUMALINE-MM1 trial started enrolling patients.

One reason the FDA would approve agents that are compared with suboptimal control arms is the subjective nature of the decision that goes into assessing the control arm. In many instances, one can construct a justification for the control arm. Moreover, suboptimal control arms may accrue events (eg, progression of disease or death) faster, leading to successful results sooner, and hastening drugs to market. Another reason involves external influences on stakeholders, such as incentives from pharmaceutical companies, the highly lethal nature of a disease, and scarcity of active agents on the market. Therefore, it is essential to design trials that are capable of being flexible. Such trials with adaptive designs use results to modify the trial’s trajectory in accordance with prespecified criteria, which in some cases, may be more efficient than traditional design.³⁵

Limitations and Strengths

Our analysis is the first of its kind that attempts to empirically evaluate the quality of control arms in clinical trials leading to marketing authorizations over a 5-year period. The main limitation of our analysis is some inherent subjectivity in the assessment of acceptable SOC therapy before a particular trial started enrolling patients. We attempted to limit subjectivity by individually and separately reviewing the literature and guidelines and establishing consensus SOC treatment for each cancer. Given that the judgment of whether a control arm is suboptimal requires interpretation, we chose to publish our results so that future investigators can directly examine the trials presented in the Table.

Conclusions

The aim of our analysis was to evaluate the quality of control arms in RCTs leading to anticancer drug approvals by the FDA. We found that, between January 1, 2013, and July 31, 2018, FDA approval of 16 (17%) of 96 anticancer drugs for the market were based on RCTs with suboptimal control arms. Future regulatory trials should be optimized to ensure that new anticancer agents being marketed are truly superior to what most clinicians would prescribe outside a clinical trial setting.

Supplement.

eTable 1. List of RCTs with a control arm that is deemed optimal

eTable 2. Nature of sub-optimal control