Enrollment Success, Factors, and Prediction Models in Cancer Trials (2008-2019)

Study on 4,004 cancer trials found 49.1% of enrolment success rate with a decreasing tendency in 2008-2019.

Abstract

PURPOSE

To investigate the enrollment success rate of cancer clinical trials conducted in 2008-2019 and various factors lowering the enrollment success rate.

METHODS

This is a cross-sectional study with clinical trial information from the largest registration database ClinicalTrials.gov. Enrollment success rate was defined as actual enrollment greater or equal to 85% of the estimated enrollment goal. The association between trial characteristics and enrollment success was evaluated using the multivariable logistic regression.

RESULTS

A total of 4,004 trials in breast, lung, and colorectal cancers were included. The overall enrollment success rate was 49.1%. Compared with 2008-2010 (51.5%) and 2011-2013 (52.1%), the enrollment success rate is lower in 2014-2016 (46.5%) and 2017-2019 (36.4%). Regression analyses found trial activation year, phase I, phase I/phase II, and phase II (v phase III), sponsor agency of government (v industry), not requiring healthy volunteers, and estimated enrollment of 50-100, 100-200, 200, and >500 (v 0-50) were associated with a lower enrollment success rate (P < .05). However, trials with placebo comparator, ≥5 locations (v 1 location), and a higher number of secondary end points (eg, ≥5 v 0) were associated with a higher enrollment success rate (P < .05). The AUC for prediction of the final logistic regression models for all trials and specific trial groups ranged from 0.69 to 0.76.

CONCLUSION

This large-scale study supports a lower enrollment success rate over years in cancer clinical trials. Identified factors for enrollment success can be used to develop and improve recruitment strategies for future cancer trials.

INTRODUCTION

Cancer is a leading cause of mortality and the main source for disease burden globally.¹ Innovations from clinical trials play a key role in the treatment of cancer. Through clinical trials, the innovations can be evaluated rigorously before possible approval and use in clinical practice, to cure the disease, prolong patient survival, and/or improve quality of life.² At the same time, eligible patients enrolled in the trials could have a chance to experience clinical benefits from the newest innovations before the regulatory approval for application in practice.

CONTEXT

• Key Objective

• To investigate the prevalence of enrollment success rate, how its trend has been over time, and what factors associated with the enrollment success rate are in cancer clinical trials.

• Knowledge Generated

• The enrollment success rate was 49.1% in 2008-2019 with a decreasing tendency over time. A set of risk and preventive factors were found for the enrollment success rate on the basis of multivariable regression models with good prediction accuracy.

• Relevance

• The low and decreasing enrollment success rate should receive a close attention by oncologists, researchers, policymakers, and other stakeholders. Our analysis leads to a prediction model on enrollment success based on the trial-level information collected prior to trial activation; the variables that are strongly associated with the enrollment success rate and meanwhile are manipulable can be used by the trial investigators as actionable strategy to improve the enrollment success rate in cancer clinical trials.

Despite the importance of cancer trials, ensuring success of patient enrollment is challenging. Studies showed nearly 35% of phase III cancer trials were closed because of insufficient enrollment.^3,4 Another study found that 18% of 4,269 phase I-III cancer trials were classified as slow enrolling, defined as less than two participants per year.⁵ Accordingly, insufficient enrollment is a major barrier to progress in clinical trials, causing a waste of both time and money, and ultimately delaying promising treatments for patients. In fact, there are sufficient number of patients eligible for participation in clinical trials but only a few of them do so.⁶

Therefore, we conduct this large-scale, cross-sectional study investigating the prevalence of enrollment success rate and its factors among clinical trials for cancers. From this work, we expect that trialists including oncologists, researchers, policymakers, and stakeholders could take more attention to enrollment success in trials for cancers, especially considering identified factors for improving enrollment in future trials.

METHODS

Data Source and Setting

This cross-sectional study was conducted based on ClinicalTrials.gov, the largest clinical trial database currently run by the US National Library of Medicine. ClinicalTrials.gov was established in 2000. The database experienced a major expansion after the Food and Drug Administration Amendment Act (FDAAA) was executed in 2007, which required all drug, biological, and device trials, besides phase I clinical trials, to be registered into the ClinicalTrials.gov database.⁷ This publicly accessible web-based registry comprises a full-scale baseline characteristic of a clinical trial, including recruiting status, disease condition, phase of the study, eligibility criteria, location, etc, allowing us to conduct such a study as generalizable as possible.

Data Extraction and Trial Selection

To extract all needed trial information from the website, we separated the progress into two parts using R (Version 3.6.1). First, we used an R library (rclinicaltrials⁸) to download all possible information. During this process, however, the extracted data did not contain two main variables, the number of patients in estimated enrollment and the actual enrollment, for calculating enrollment success rate, the outcome of this study. Therefore, we conducted the second part of the extraction using web scraping methods in R to extract both variables.

Data extraction was completed on June 6, 2022, on the basis of the trial activation year of 2008-2019. The selection for the start year (2008) was in concordance with the FDAAA introduced in 2007, so that trials started after 2008 should have completer and more robust information as recorded in the registration website. The selection for the ending year (2019) was given that many trials started after 2019 are still ongoing, without enrollment completion; particularly, including those impacted by the COVID-19 pandemic.^9-12 Breast, lung, and colorectal cancers were selected according to their disease burden—all of them are of the top five cancer types with the highest mortality rate and disability-adjusted life-years among all cancer types in the globe.¹ Exclusion criteria were trials with completion date of 2020 and afterward; trials with no information in the Study Information dataframe, which contains key variables including trial activation date, number of patients in estimated enrollment, and actual enrollment; trials whose study type is not interventional; and trials with undefined enrollment names and unknown (marked as NA) phase status.

Outcome

Our outcome is enrollment success. It is a binary variable, defined as actual patient enrollment is ≥85% of the estimated patient enrollment goal of the trial (Appendix Table A1, online only). The rationale for using 85% as the cutoff value is that we could not conclude enrollment unsuccessful if the actual enrollment number were below the estimated number by a few participants. The cutoff of 85% gives the outcome definition with an acceptable margin to avoid the mentioned issue. According to the definition, we calculated the prevalence of enrollment success as the number of trials with enrollment success divided by the total number of included trials.

Exposure

To investigate the factor for enrollment success, the following trial characteristics were used as exposure: trial activation year, cancer type, number of conditions, phase, intervention type, number of interventions, type of intervention drug, lead sponsor agency, number of sponsors, eligibility of healthy volunteers, minimum age and maximum age, type of arms, number of arms, number of primary outcomes, number of secondary outcomes, number of countries, number of locations, and number of patients in estimated enrollment. In addition, recruitment status was extracted.

For the lead sponsor agency, the website has four categories: NIH, US Fed, industry, and other. We reclassified the categories by combining NIH and US Fed as government, and subdividing other into research institute and others by searching for keywords (“University,” “Center,” “Institute,” “Group,” “Hospital,” and “Network”) that could distinguish most of the research institutes from all the sponsors. For the type of intervention drug, we chose the top five most frequent types in the database.

Statistical Analysis

The association of trial characteristics with enrollment success (yes or no) was first evaluated via univariate analyses, using the chi-square test for categorical variables and the Kruskal-Wallis test for continuous variables. The association was further evaluated via the multivariable logistic regression model, adjusted for all the exposure variables mentioned above. The measure is the odds ratio: the point estimate and 95% CIs <1 indicate risk factors lowering the enrollment success rate. The above model was conducted for all included trials, trials by cancer type (breast cancer, lung cancer, and colorectal cancer) and by phase (phase I, phase II, phase III), as well as trials for drugs only. Last, the prediction accuracy for each model was measured by the area under the receiver operating curve (AUC).

RESULTS

From a total of 24,886 trials identified in the database for breast, lung, and colorectal cancers, 4,004 trials were included (Fig 1). Appendix Table A2 presents the details of trial characteristics. Among all trials, 37.2%, 32.6%, and 15.5%, respectively, were for breast, lung, and colorectal cancers only, the rest were the trials for multiple tumor types including any of the three cancers; 26%, 44.2%, and 13.7% were phase I, phase II, and phase III trials, respectively; 11.9%, 32.0%, 41.5%, and 14.6% were mainly sponsored by government, research institutes, industry, and others, respectively. Appendix Tables A3 and A4 present the characteristics by cancer type and by phase, respectively; Appendix Table A5 is for drug trials.

FIG 1.

Flowchart of trial selection. NA, not applicable.

Prevalence of Enrollment Success Rate

The prevalence and the trend of enrollment success rate are shown in Figure 2. The overall enrollment success rate for all included trials was 49.1%. Compared with 2008-2010 (51.5%) and 2011-2013 (52.1%), enrollment success rate was lower in 2014-2016 (46.5%) and 2017-2019 (36.4%). The decreasing trend exists regardless of trials by cancer type and lead sponsor agency, as well as in phase I, phase II, biological, drug, and radiation trials, respectively. The enrollment success rate in phase III trials remained stable over time: 63.9% in 2008-2020, 64.1% in 2011-2013, 69.2% in 2013-2016, and 60.6% in 2017-2019. The rate in trials sponsored by government was low: 37.9% in all years, and 43.0%, 35.7%, 33.3%, and 21.7% in 2008-2010, 2011-2013, 2014-2016, and 2017-2019, respectively.

FIG 2.

Enrollment success rate in clinical trials: (A) all trials, (B) by cancer type, (C) by phase, (D) by intervention type, (E) by drug type, and (F) by lead sponsor type. TKI, tyrosine kinase inhibitor.

Factors for Enrollment Success Rate

Table 1 presents the multivariable logistic regression results on the associations between trial characteristics and enrollment success in all trials and trials for breast, lung, and colorectal cancers; Appendix Table A6 presents the results in phase I, phase II, phase III, and drug trials. For all trials, risk factors were trial activation year; phase I, phase I/phase II, phase II, and phase II/phase III trials, compared with phase III trials; radiation trials; trials with two interventions, compared with trials with 1 intervention; lead sponsor agencies of government and other, compared with industry; trials not requiring healthy volunteers; trials with patients recruited from an unknown number of countries, compared with one country; and estimated enrollment of 50-100, 100-200, 200-500, and >500 patients, compared with 0-50 patients. The risk factors of trial activation year, phase I, phase I/phase II, and phase II, sponsor agency of government, not requiring healthy volunteers, and estimated enrollment of 50-100, 100-200, 200, and >500 patients were still found in majorities of trials by cancer type and phase (I, II, III; Appendix Table A6) as well as drug trials (Table 1; Fig 3).

TABLE 1.

Factors for Enrollment Success Rate in All Included Clinical Trials and Trials by Cancer Type

FIG 3.

Forest plots for presenting selected factors for enrollment success rate. Direction of OR toward 0 from 1 indicates risk factor; on the contrary, direction from 1 to larger indicates preventive factor. OR, odds ratio.

Preventive factors associated with a higher enrollment success rate in all trials were trials for colorectal cancer; trials for device; randomized trials, compared with nonrandomized trials; trials with active comparator and placebo comparator arms; trials with ≥3 arms, compared with two arms; trials with two and ≥5 secondary end points, compared with 0 secondary end points; and trials with ≥5 locations, compared with one location (Table 1). The factors of trials with placebo comparator, ≥5 locations, and trials with ≥5 secondary end points were also found in majorities of trials by cancer type and phase (I, II, III; Appendix Table A6) as well as drug trials (Table 1; Fig 3).

Model Prediction

The prediction score, AUC, of the logistic regression model for all trials was 0.69. The AUC for breast, lung, colorectal, phase I, phase II, phase III, and drug trials was 0.71, 0.74, 0.74, 0.72, 0.69, 0.76, and 0.70, respectively.

DISCUSSION

This is a large-scale, cross-sectional study investigating the prevalence of enrollment success rate and its risk and preventive factors among clinical trials for breast, lung, and colorectal cancers—the ones accounting for the highest disease burden among all cancer types. Overall, the study estimated the enrollment success rate as <50% among all the included trials in 2008-2019; specifically, the trend presented was decreasing over time, with 36.39% in 2017-2019. Also, the study identified risk and preventive factors for enrollment success rate. Important risk factors included trial activation year, phase I, phase I/phase II, and phase II (v phase III), sponsor agency of government (v industry), not requiring healthy volunteers, and estimated enrollment of 50-100, 100-200, 200, and >500 (v 0-50); preventive factors included placebo comparator, ≥5 locations (v one location), and a higher number of secondary end points (eg, ≥5 v 0). The final regression models for investigating the above associations present a decent prediction, with AUC in all trials and trials by characteristic ranging from 0.69 to 0.76.

Our study highlights the undesirable enrollment success rate in clinical trials for cancers, especially its decreasing tendency. Of note, these findings were based on the trials before the start of COVID-19 pandemic. Therefore, we hold the concern about enrollment success rate in the currently ongoing trials, especially given the published evidence supporting the difficulties in trial enrollment due to the COVID-19 pandemic.^9-12 Continuous investigations on enrollment success are needed whenever more robust data could be available for the trials started or completed during the COVID-19 years. Nevertheless, among the results for different trials, the enrollment success rate in phase III trials was high—ranging from 60.6% to 69.2% throughout the years of 2008-2019—which echoes the results in two previous studies on phase III trials in the same years of 1993-2002: 63% of enrollment success rate in 248 trials sponsored by the US National Cancer Institute,⁴ and 66% in 238 trials of the US Clinical Trials Cooperative Groups.³

Regarding risk factors, specifically, the lower enrollment success rates in phase I and II trials should be of concern. This is given that the trials in both phases are the cornerstone of developing phase III trials from which, in general, the innovations could be considered for application in real-world practice. As such, enough patient enrollment ensures the rigor before being considered for and evaluated in phase III trials. Furthermore, we are surprised to see the difference in the enrollment success rate between lead sponsor agencies. Specifically, the rate in government-sponsored trials is very low, decreasing from 43.0% in 2008-2010 to 21.7% in 2017-2019. Such results should be scrutinized by the public, as the investment from government and the decision for such investment are made in collaboration with well-recognized scholars.

Last, the estimated enrollment of over 50 patients is a risk factor; as more patients are expected to be enrolled, there would be higher cost and difficulty in trial conduction. In addition, such results in the majority of these models suggest a higher number of estimated patients leads to a higher risk that the trial could not achieve the enrollment target number. In addition to the above risk factors, structural and clinical barriers have been well studied.^13,14 For example, a trial might not be available at the collaborating centers; patients may be ineligible for an available trial; and enrollment rates are different between academic and community settings.¹³ Other common causes of enrollment failure are narrow eligibility criteria, overestimate of targeted population, lack of willingness, training, and engagement among medical practitioners, recruiters, and patients, and insufficient funding.^15-18

Correspondingly, here we discuss three feasible approaches to facilitate enrollment success in clinical trials. First, it is pragmatic to expand the number of collaborating centers for a trial. The expansion is supported by our study, showing that trials with five or more locations were twice as likely to achieve enrollment success than the trials with one location. Other approaches to improve enrollment are related to trial design: carefully expanding eligibility criteria¹⁹ as well as lowering the sample size by using validated surrogate end point(s).^20-22 The former way has been debated for decades but is well supported by real-world evidence that many patients not eligible under the original trial could potentially benefit from the treatments.²³ At the same time, the expansion could improve trials' generalizability to a broader population, and help address the issues of disparities in access to and participation in clinical trials, in terms of race, sex, age, or other demographic and disease characteristics, as strongly supported by considerable studies.^17,24-34 Regarding the use of surrogate end points, its implementation is also helpful to reduce trial conduction time to save 11-19 months compared with trials using overall survival.³⁵ To safely and effectively implement the last two approaches as mentioned, we echo the need of strengthening the use of real-world studies and postmarket trials as guidance.^23,24,36-38

This is one of the largest studies investigating enrollment success rates among clinical trials for breast, lung, and colorectal cancers. Relying on ClinicalTrials.gov, the study captured a large variety of clinical trial information for the cancers, applied the information to the prevalence and trend of enrollment success rate, evaluated its factors, and developed prediction models for enrollment success rate for the included trials. Regarding limitations, however, this study is strongly subjective to the robustness of data collection on ClinicalTrials.gov. Specifically, although ClinicalTrials.gov is the largest database for trial registration, our sample of trials cannot represent the whole of all trials registered and not registered in the globe, especially those conducted not in the United States. Also, the data quality and completeness depend on the trialists who submitted the trial data. Regarding the submission, there could be a time lag between actual trial conduction and submission, which is why we failed to include ongoing trials and trials activated in the most recent years (after 2019) because of a large amount of missing data on trial information, especially the variables used for constructing the outcome variable (enrollment success rate). Otherwise, we would look at how the COVID-19 pandemic has affected enrollment success in cancer clinical trials, which deserves future investigations.

In addition, our analysis was also subject to the availability of data information developed on ClinicalTrials.gov. Rather than patient level or health system level, specifically, the analysis was based on the information at the trial level but the amount of analysis should provide informative implications as presented and discussed in the Result and Discussion sections. Last, the decreasing tendency found in our study could be accounted for by the more registrations of clinical trials in ClinicalTrials.gov, including those without successful enrollment. However, the decreasing trend is consistent with the findings from other studies in the high failure rate of oncology trials because of low enrollment rate.^15,17,39,40 In fact, there are increasing number of new therapies, especially with the development of immunotherapy,^41,42 to be evaluated, and persistent barriers to enrollment and trial completion.^{13-18,39,40,43} As such, the low enrollment success rate found over the past decade should be paid attention to and be potentially improved by considering the factors as identified in this study.

In conclusion, this large-scale, cross-sectional study supports a lower enrollment success rate over years in cancer clinical trials, regardless of cancer types and lead sponsor agencies, and trials by many other characteristics. The findings could be of concern to the public, especially trialists including oncologists, researchers, policymakers, and other stakeholders. Our analysis leads to a model with good prediction on enrollment success on the basis of the trial-level information collected before trial activation. The identified risk factors for enrollment success can be used to develop and improve recruitment strategies for future cancer trials.

APPENDIX

TABLE A1.

Enrollment Name Combinations in ClinicalTrials.gov

TABLE A2.

Characteristics of Included Trials

TABLE A3.

Characteristics of Included Trials by Cancer Type

TABLE A4.

Characteristics of Included Trials by Phase

TABLE A5.

Characteristics of Drug Trials

TABLE A6.

Factors for Enrollment Success Rate in Phase I, Phase II, Phase III, and Drug Clinical Trials

AUTHOR CONTRIBUTIONS

Conception and design: Jianrong Zhang, Herbert Pang, Thomas E. Stinchcombe, Xiaofei Wang

Administrative support: Xiaofei Wang

Collection and assembly of data: Siqi Zhang, Sida Liu, Xiaofei Wang

Data analysis and interpretation: Siqi Zhang, Jianrong Zhang, Herbert Pang, Thomas E. Stinchcombe, Xiaofei Wang

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Enrollment Success, Factors, and Prediction Models in Cancer Trials (2008-2019)

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/op/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).