Premature Clinical Trial Discontinuation in the Era of Immune Checkpoint Inhibitors

Early trial termination or withdrawal is common and impairs progress. Using ClinicalTrials.gov, the largest publicly available clinical trial registry, the frequency and reasons associated with early trial discontinuation were examined.

Abstract

Background.

Clinical trial completion is critical for new cancer therapies. Premature trial termination or withdrawal is common and impairs progress. We assessed factors of early terminated/withdrawn oncology trials focusing on trials with immune checkpoint inhibitors (ICI), hypothesizing that the latter may be associated with lower rates of premature discontinuation.

Materials and Methods.

We reviewed all adult, intervention, oncology trials registered in ClinicalTrials.gov (November 16, 2011, to April 16, 2015) to identify all terminated/withdrawn trials and reasons for termination. Logistics regression model was used to identify factors associated with early termination/withdrawal. Discontinuation rate was compared in trials with and without ICI.

Results.

We identified 12,875 trials (35% industry funded, 12% federal funded), of which 8.5% were prematurely terminated (5%) or withdrawn (3.5%); the main reasons were poor accrual (33%) and logistical (24%). ICI trials (n = 350) had a nonsignificant lower rate of termination or withdrawal compared with all other oncology trials (5.4% vs. 8.5%; p = .9) and were less likely to discontinue due to poor accrual (nonsignificant difference: 21% vs. 33%; p = .4). ICI trials were also less likely to discontinue compared with all other oncology drug trials (e.g., chemotherapy, targeted inhibitors, antiangiogenesis, biologics; 5.4% vs. 7.9%, respectively, nonsignificant difference). The 4‐year cumulative incidence of failing to complete for reasons unrelated to toxicity or efficacy was 18% (95% confidence interval 16%–20%). There was no association between annual incidence across different tumor types or accrual goal and rate of trial termination.

Conclusion.

Poor accrual represents the main cause of early cancer trial termination. Premature termination/withdrawal rate was not significantly lower in ICI compared with other trials. Clinical trial completion remains a high priority and can be influenced by provider and patient factors.

Implications for Practice.

Clinical trial completion is critical for new cancer therapies. Premature trial termination or withdrawal is common and impairs progress. This study assessed factors of early terminated/withdrawn oncology trials, focusing on trials with immune checkpoint inhibitors (ICI), and found that poor accrual represents the main cause of early cancer trial termination. Premature termination/withdrawal rate was not significantly lower in immune checkpoint inhibitor trials compared to other trials. The discussion herein is focused on measures taken by the National Cancer Institute and other institutions to improve clinical trial accrual and prevent premature clinical trial discontinuation.

Introduction

Clinical trials are essential in generating evidence to guide clinical oncology practice and improve outcomes. They often study novel interventions, which may be toxic or clinically ineffective, and hence may be terminated based on prespecified interim analysis results. However, a significant proportion of clinical trials fail to achieve completion due to insufficient accrual or other reasons [1]. This can result in significant loss of resources involved in designing and setting up trial infrastructure, capturing and analyzing data, as well as recruiting patients, without contributing meaningfully to the scientific knowledge and evidence base. Cheng et al. reported that 37.9% of phase III clinical trials approved by the National Cancer Institute's (NCI) Cancer Therapy Evaluation Program failed to achieve minimal accrual goal [2]. Previous studies have reported that less than 5% of adult cancer patients enroll in clinical trials [3]. This is in stark contrast to the fact that 70% of U.S. residents are interested in participating in clinical trials, indicating a huge gap between actual trial accrual and patient willingness to participate [4]. Numerous studies have tried to identify barriers of poor trial accrual [5], [6], [7], [8]; it seems that physician‐associated reasons are common. Somkin et al. concluded that oncologists’ belief and trust in the treatment approach is an important determinant for patient clinical trial participation [9]. Immunotherapies such as immune checkpoint inhibitors (ICI) have shown very significant antitumor activity and a relatively favorable safety profile and have been approved for certain clinical indications, while being tested for others in numerous clinical trials worldwide [10]. Market forecasts have projected that ICI will dominate the immunotherapy market soon and capture a substantial 85% market share in 2022 [11]. The overall market for ICI peaks at nearly $35 billion a year; therefore, the degree of enthusiasm by both providers and patients is very high and may affect their interest in ICI trials.

We hypothesized that the discontinuation rate is lower in ICI trials compared with all other cancer trials, and that accrual rate may account for such difference. Stensland et al. studied the rate of cancer clinical trial termination between 2005 and 2011 and concluded that the cumulative incidence of cancer clinical trials failing to complete for reasons other than toxicity and efficacy was 20% [12]. We sought to determine the frequency and reasons associated with early trial discontinuation for all trials registered between 2011 and 2015, to understand the differences in rates of termination in all other and ICI trials. We used ClinicalTrials.gov, the largest publicly available clinical trial registry, to determine the frequency and reasons for premature clinical trial termination.

Materials and Methods

Eligible Clinical Trials

We searched ClinicalTrials.gov on August 18, 2016, to identify all randomized controlled clinical trials registered between November 16, 2011, and April 16, 2015, using the search terms “neoplasm,” “cancer,” and “tumor.” The search strategy is defined in Figure 1. The search was restricted to adult population (18 years and above) and interventional studies. Trials designated as “terminated” (stopped recruiting participants prematurely and are not likely to start again) and “withdrawn” (stopped before enrolling any patients) represent trials failing to complete; therefore, these were included in our analysis (supplemental online Appendix 1). All trials with a start date before November 16, 2011, or after April 16, 2015, trials with “suspended” status (may start again), and trials in pediatric patients (<18 years) were excluded. ICI trials included cytotoxic T‐lymphocyte‐associated antigen 4 inhibitors (ipilimumab, tremelimumab), programmed cell death protein 1 inhibitors (nivolumab, pembrolizumab), and programmed death‐ligand 1 inhibitors (atezolimumab, durvalumab, avelumab). All included trials were independently reviewed by two authors (M.K., S.R.). Two reviewers (M.K., S.R.) independently extracted data from trials using standardized data extraction sheets, and all discrepancies were resolved in consensus.

Figure 1.

ClinicalTrials.gov search strategy.

Data Characteristics

The following information was extracted for each clinical trial: study population, cancer type, start and completion dates, intervention type, source of funding, trial phase, trial location, and current recruitment status. The data elements are defined in Table 1. For terminated and withdrawn trials, reasons for trial discontinuation were collected and tabulated independently by two investigators (M.K. and S.R.) based on data provided in ClinicalTrials.gov registry. Reasons were divided into multiple categories: poor accrual, lack of efficacy, toxicity (side effects/adverse events), logistical reasons (e.g., insufficient funding, lack of study drug, principal investigator left institution, cancellation by sponsor), and inadequate study design/trial not necessary (proven efficacy/existence of a similar trial making the current trial unnecessary).

Table 1. Characteristics of adult cancer clinical trials from ClinicalTrials.gov 2011–2015 included in the analysis (n = 12,875 for all trials; n = 350 for ICI trials).

Abbreviation: GI, gastrointestinal; GU, genitourinary; ICI, immune checkpoint inhibitors; NA, not applicable.

Statistical Analysis

Multivariate Cox proportional hazards regression models were used to evaluate study characteristics independently associated with failure to complete, including ICI trial, trial phase, and funding source, and associations were expressed as hazard ratios (HR) and their 95% confidence intervals (CI). We calculated the overall cumulative incidence of clinical trials failing to complete for reasons other than safety and efficacy and subsequently determined the failure to completion rate as the inflection point at which the cumulative incidence of termination no longer changed, that is, when the slope of the cumulative incidence curve approached zero, as described by Stensland et al. [12]. A subgroup analysis was also performed to determine the cumulative incidence of clinical trials specifically terminated due to poor accrual. Annual incidence of different cancers was obtained from the Surveillance, Epidemiology and End Results Program of NCI [13]. Annual incidence across different tumor types was correlated with rates of trial termination. We used R 3.3.2 (www.r-project.org) for all statistical analyses.

Results

Table 1 shows the characteristics of the 12,875 included clinical trials identified in the study period, of which 1,091 failed to complete: 692 (5%) were early terminated and 399 (3.5%) withdrawn, respectively (Fig. 2). Table 2 summarizes the characteristics of all early terminated and withdrawn trials. The reasons for early discontinuation for all ICI and all other cancer clinical trials are represented in Figure 3; the most common reason was poor accrual (33%), followed by logistical reasons (24%) among all other cancer clinical trials.

Figure 2.

Recruitment status of all cancer clinical trials and ICI trials.

Abbreviation: ICI, immune checkpoint inhibitors.

Table 2. Characteristics of early terminated and withdrawn clinical trials.

Abbreviations: ICI, immune checkpoint inhibitors; NA, not applicable.

Figure 3.

Reasons for early termination/withdrawal.

Abbreviation: ICI, immune checkpoint inhibitors.

In subgroup analysis of ICI trials, there were 350 ICI trials during the study period, of which 14 (4%) were terminated and 5 (1.4%) withdrawn (Fig. 2). The reasons for ICI trial termination/withdrawal and comparisons to all other cancer trials are represented in Figure 3. The ICI trials appeared (numerically) less likely to discontinue due to poor accrual compared with all other cancer clinical trials (21% vs. 33%); however, the results were not statistically significant. ICI trials were also less likely to discontinue when compared with all other oncology drug trials (including trials with chemotherapy, targeted inhibitors, antiangiogenesis, and biologics; 5.4% vs. 7.9%, respectively); however, the difference was not statistically significant. Multivariate Cox regression analyses showed individual institution‐sponsored trials were significantly more likely to discontinue compared with industry‐sponsored trials (HR = 1.54, 95% CI 1.33–1.78; p < .0001). Phase II and phase III trials were significantly less likely to discontinue compared with phase I trials (HR = 0.87, 95% CI 0.75–1.0; p = .05; HR = 0.28, 95% CI 0.21–0.38; p < .001, respectively).

We did not find a significant correlation between rate of termination/withdrawal and cancer annual incidence (Spearman correlation Rho = −0.32, p = .37; supplemental online Fig. 1). We also compared the median enrollment/accrual between terminated trials and completed trials using the Wilcoxon rank sum test. The accrual goal was higher for completed trials (n = 2,717) with a median 47 (25th, 75th percentiles: 22, 111) compared with terminated trials (n = 695) with a median 9 (25th, 75th percentiles: 3, 28; p < .0001). The 4‐year cumulative incidence of trials failing to complete for reasons unrelated to toxicity or efficacy was 18% (95% CI 16%–20%; Fig. 4). Median (interquartile range) follow‐up for failure to complete for any reason was 18.5 (8.1–32) months. The 4‐year cumulative incidence of failing to complete because of poor accrual was 6% (95% CI 4%–8%; Fig. 5).

Figure 4.

Cumulative incidence of clinical trial premature termination/withdrawal for any reason (unrelated to toxicity or lack of efficacy).

Figure 5.

Cumulative incidence of trials failing to complete due to poor accrual.

Discussion

Our study demonstrated that between 2011 and 2015, 8.5% of all adult, intervention oncology clinical trials closed prematurely, two thirds of which had already enrolled participants at the time of trial termination. Poor accrual and logistical issues with the conduct of the trial were the most common reasons for trial discontinuation, whereas only 10% of the trials were discontinued due to toxicity/lack of efficacy. Cumulative incidence of clinical trial termination for reasons other than safety and efficacy between 2011 and 2015 was determined to be 18%, which is slightly lower as compared with the previously reported 20% between 2005 and 2011 [12]. Similarly, cumulative incidence of clinical trials terminated due to poor accrual was 6% in 2011–2015, lower compared with 9% reported between 2005 and 2011 [12].

Over the last decade, both NCI and the American Society of Clinical Oncology have made several efforts to improve clinical trial accrual and completion; for instance, they cosponsored the Cancer Trial Accrual Symposium in April 2010, to provide recommendations for improving trial recruitment [14]. Subsequently, NCI launched AccrualNet, a searchable database of hundreds of journal articles with the main aim of helping clinical trial professionals in recruiting and retaining clinical trial participants, attaining accrual goals, and achieving primary endpoints [15]. In 2011, NCI Community Cancer Centers Program (NCCCP) developed the clinical trial best practice matrix tool, now renamed as Clinical Trial Assessment of Infrastructure Matrix, to facilitate research program improvements through annual self‐assessments and benchmarking [16]. The tool identified nine attributes, each with three subsequent levels, to score clinical trial infrastructural elements. Subsequently, in 2016, Hirch et al. concluded that with the use of this tool, trial availability improved more quickly at NCCCP sites compared with national trends (30% at NCCCP vs. 16% nationally) and accrual increased 30% nationally compared with 133% across NCCCP sites [17].

Our study showed a significantly higher rate of trial termination across trials sponsored by individual institutions as compared with industry and federally sponsored trials. Sponsorship by academic institutions has unique challenges, including time‐limited funding with substantial proportion for inflexible commitments (e.g., personnel salaries). In such a situation, if rate of enrollment is slow and the trial period prolonged, further funding may not be available and accrual goal may not be met [18]. This calls for a need to improve trial accrual in academic institutions. Several such efforts have already been made. For instance, Trial Prospector, a point‐of‐care application for screening patients for clinical trials, was developed by Case Comprehensive Cancer Center (Cleveland, OH) and was shown to have a positive role for improving clinical trial accrual [19]. Other institutions have tried issuing Clinical Trial Infrastructure Funds, which improved clinical trial accrual by 12%–13% from baseline [20].

Comparison between all other and ICI trials yielded nonsignificantly lower rates of discontinuation in the latter. Overall, a quarter of discontinued ICI trials did so because the trial was no longer deemed necessary. Such early discontinuation may prevent further waste of finite resources and may be unavoidable at a certain rate. Moreover, poor accrual was numerically less likely to be a cause for trial termination in ICI trials compared with all other oncology trials, despite lack of statistical significance. Our assumption is that both physician‐ and patient‐driven behaviors and attitudes combined with the vast new literature and social media coverage may contribute to the accrual of ICI trials and may suggest that accrual in general could be improved by targeted educational interventions and focused efforts.

Completing clinical trial accrual in a timely manner is especially important also in the era of “precision oncology,” as both “umbrella” and “basket” types of trials are being conducted based on the presence of putative “driver” genomic alterations that can represent treatment targets and/or predictive biomarkers. Moreover, future reassessment of the rate and reasons of premature clinical trial discontinuation may provide additional information and keep engaging multiple stakeholders. Finally, studies examining the costs associated with early clinical trial discontinuation are required to provide insight into the associated economic burden.

Our study has several limitations that merit consideration. In order to make our study comprehensive and facilitate comparison between all other and ICI trials, we included phase I trials; however, it must be noted that registration of phase I clinical trials at ClinicalTrials.gov was not made mandatory by the U.S. Food and Drug Administration Amendment Act, 2007 [21]. However, this should not significantly affect our study, as the International Committee of Medical Journal Editors issued a statement in 2005 mandating registration of clinical trials as a prerequisite for publication [22]. In addition, documenting a reason for termination/withdrawal is not a mandatory field in ClinicalTrials.gov, suggesting a possibility of selection bias in the analysis. There is unmeasured confounding bias inherent to the study nature that may influence interpretation of our findings; however, this study does reveal a snapshot of the clinical trial accrual landscape and may further trigger constructive discussions among several stakeholders.

Implications of Our Study for Clinicians and Policymakers

The advent of ClinicalTrials.gov has provided the necessary database for monitoring the progress of clinical trials. However, efforts should be made to document the reasons for each clinical trial discontinuation in order to better assess them as well as trigger the development of potential interventions. Combined with previous studies of early trial discontinuation [12], [23], [24], this study shows that progress has been made in successful conduct and completion of clinical trials; however, the issue of early trial discontinuation remains important. Accrual in clinical trials remains a high priority and requires collaborative efforts by several stakeholders.

See http://www.TheOncologist.com for supplemental material available online.

Author Contributions

Conception/design: Monica Khunger, Sagar Rakshit, Adrian V. Hernandez, Vinay Pasupuleti, Kate Glass, Matthew D. Galsky, Petros Grivas

Provision of study material or patients: Matthew D. Galsky, Petros Grivas

Collection and/or assembly of data: Monica Khunger, Sagar Rakshit

Data analysis and interpretation: Monica Khunger, Sagar Rakshit, Adrian V. Hernandez, Vinay Pasupuleti, Kate Glass, Matthew D. Galsky, Petros Grivas

Manuscript writing: Monica Khunger, Sagar Rakshit, Adrian V. Hernandez, Vinay Pasupuleti, Kate Glass, Matthew D. Galsky, Petros Grivas

Final approval of manuscript: Monica Khunger, Sagar Rakshit, Adrian V. Hernandez, Vinay Pasupuleti, Kate Glass, Matthew D. Galsky, Petros Grivas

Disclosures

The authors indicated no financial relationships.