Bayesian Clinical Trials at The University of Texas MD Anderson Cancer Center: An Update

Abstract

Background/Aims:

In our previous article (Biswas, et al. Clinical Trials 2009), we showed that Bayesian methods had established a foothold in developing therapies in our institutional oncology trials. In this article, we will document what has happened since that time. Additionally, we will describe barriers to implementing Bayesian clinical trials, as well as our experience overcoming them.

Methods:

We reviewed MD Anderson Cancer Center clinical trials submitted to the institutional protocol office for scientific and ethical review between January 2009 and December 2013, the same length time period as the previous article. We tabulated Bayesian methods implemented for design or analyses for each trial, and then compared these to our previous findings.

Results:

Overall, we identified 1020 trials and found that 283 (28%) had Bayesian components so we designated them as Bayesian trials. Among MD Anderson-only and multi-center trials, 56% and 14%, respectively were Bayesian, higher rates than our previous study. Bayesian trials were more common in Phase I/II trials (34%) than in Phase III/IV (6%) trials. Among Bayesian trials, the most commonly used features were for toxicity monitoring (65%), efficacy monitoring (36%), and dose finding (22%). The majority (86%) of Bayesian trials used non-informative priors. Seventy-five (27%) trials applied Bayesian methods for trial design and primary endpoint analysis. Among this latter group, the most commonly used methods were the Bayesian logistic regression model (N=22), the continual reassessment method (N=20), and adaptive randomization (N=16). Median Institutional Review Board approval time from protocol submission was the same 1.4 months for Bayesian and non-Bayesian trials. Since the previous publication, the Biomarker-Integrated Approaches of Targeted Therapy for Lung Cancer Elimination (BATTLE) trial was the first large scale decision trial combining multiple treatments in a single trial. Since then, two regimens in breast cancer therapy have been identified and published from the cooperative Investigation of Serial Studies to Predict Your Therapeutic Response with Imaging and Molecular Analysis (I-SPY 2 TRIAL), enhancing cooperation among investigators and drug developers across the nation, as well as advancing information needed for personalized medicine. Many software programs and Shiny applications for Bayesian trial design and calculations are available from our website which has had more than 21,000 downloads worldwide since 2004.

Conclusions:

Bayesian trials have the increased flexibility in trial design needed for personalized medicine, resulting more cooperation among researchers working to fight against cancer. Some disadvantages of Bayesian trials remain, but new methods and software are available to improve their function and incorporation into cancer clinical research.

Background/Aims

Practical applications of Bayesian oncology clinical trials began in 1985 with pharmacokinetics analysis software^{1, 2} followed by clinical trial designs. From the early to mid-1990’s, several influential papers on applying Bayesian methods to cancer trials were published. Methods ranged from phase I trials including continual reassessment method³ and escalation with overdose control⁴ to screening efficacious agents in phase II trials and designing trials with multiple outcomes to interim monitoring and early stopping of trials.^3–16

Since the 1990’s, the use of Bayesian methods in cancer clinical trials has expanded. Although the number of publications on Bayesian methods in clinical trials has increased, their actual applications are still limited.^17–19 Even with growing awareness and methodology breakthroughs, many barriers still remain for their use.²⁰ Stepping stones to implementing Bayesian designs to overcome these barriers include eliciting a meaningful prior, having suitable software for real-time decision-making and/or simulating complex designs, increasing a desire among biostatisticians and clinicians to make the extra effort to implement newer designs, improving the quality of reporting of Bayesian methods, and support from regulatory agencies.^17–21 Despite the fact that many novel Bayesian trial methods have been proposed, they only have impact when used. We previously reported our experience at the MD Anderson in 2009, in which 964 treatment protocols were reviewed between the years 2000 to 2005.²²

The goals of this paper are to provide an update to our prior findings, characterize the use of Bayesian methods in cancer clinical trials, and identify potential barriers and challenges for implementing these methods. Here we expand our previous institutional experience by reviewing protocols registered from January 2009 to December 2013, in the same length subsequent time period. We have added details of specific statistical methods and endpoints described in the protocol, and compare these with our previously published experience.²² Additionally, we explore whether trials with Bayesian designs show any differences in institutional review board (IRB) approval and trial activation times.

We also walk through examples of impactful Bayesian trials that took novel approaches to identify potential new treatments and combinations in lung cancer^23–26 and breast cancer.^27–29 Finally, we discuss the observed changes and strategies for overcoming certain common barriers to implementing these flexible designs, in an effort to illuminate solutions for investigators to implement Bayesian methods in their clinical trials.

Methods

Inclusion/exclusion criteria

We collected information on all treatment clinical trials that were submitted from January 2009 to December 2013 for review by the MD Anderson Clinical Research Committee and Institutional Review Board (IRB). We removed duplicate protocols, those withdrawn during the review process prior to data collection, or any terminated prior to activation. Trials that were submitted and not yet activated at the time of data collection are included.

Data collection

The institutional Office of Protocol Research maintains a database of basic protocol information. All protocols with submission dates between January 2009 and December 2013 were collected on October 1, 2014. Information from this source included protocol number, principal investigator, department, trial phase, study source, trial site, disease group, planned total accrual, and planned and actual MD Anderson accrual, as well as dates of submission, IRB approval, and activation.

A second source of information is a database of all protocols reviewed within the Department of Biostatistics, which contains information on review dates, the collaborating statistician for institutional trials, and the presence of Bayesian methods in the protocol. The department reviews all treatment research protocols, except for cooperative group trials.

A third source is the Department of Biostatistics’ Clinical Trial Conduct website. This site implements several common Bayesian and non-Bayesian methods in real time in a protected, encrypted environment accessible directly to medical research personnel. It allows for randomization and/or monitoring of trials implementing several Bayesian or other adaptive methods, with quality checks for up-to-date information in clinical measures needed for the calculations. From this site, we identified implementation of Bayesian and non-Bayesian designs for the subset of protocols using its services.

The information from these three sources was merged by protocol ID to create a master list of protocols that were submitted for scientific and ethical review during the applicable time frame. With this list, protocol documents were then individually reviewed to extract gaps in existing information, correct any conflicts, and identify and collect statistical design and methods by two analysts (AP and MC). Quality control checks were performed by the team with resolution made by a third analyst (RSST). Bayesian methods were identified by the language within the statistical methods section, for example use of the word Bayesian, presence of prior distributions incorporated into planning, calculations of posterior distribution in the methods, or if a referenced method not described in the statistical methods was found to be Bayesian upon review. When Bayesian methods were identified, the specific designs and methods were recorded. Additionally, the protocol was classified by whether or not the trial had a Bayesian primary design (i.e., was both designed and analyzed as Bayesian). Bayesian trials without a Bayesian primary design had Bayesian interim monitoring incorporated but used frequentist methods for their primary endpoint or design. They may or may not have used Bayesian methods for secondary or exploratory endpoints. Additionally, we determined how the Bayesian method was implemented throughout the study. For example, the trial had pre-calculated boundaries printed in the protocol such as the maximum number of patients with toxic events in cohorts of 5 patients for safety monitoring, or needed active analysis by a statistician at each look requiring data acquisition and calculations reported back to the clinical research team at each decision point. If multiple Bayesian methods were included then different implementations may have been made (e.g., different approaches for safety monitoring and futility analyses). When different implementations were identified, the trial is categorized into a single category in the following priority order: 1) implemented on the Clinical Trial Conduct website described above; 2) performed by the statistician throughout the study; 3) written in the protocol with pre-calculated boundaries; or 4) unspecified/other. This review also determined features such as the number of treatment arms. Patients receiving different treatments or different doses of the same treatment were on different arms. Patients with different disease cohorts receiving the same treatment were not considered separate arms.

Statistical analysis

Descriptive statistics are reported to summarize the trial characteristics. Protocol features including submission year, study initiation site, funding source, planned and actual accrual, and protocol status as of October 1, 2014 are tabulated overall and by whether the protocol implemented Bayesian methods. Statistical design and endpoints are tabulated by the presence of Bayesian methods and trial phase. For trials with Bayesian methods, we tabulated the number of methods, whether or not an informative prior was used, which methods were implemented, and how they were conducted. The numbers and percentage of Bayesian trials by phase are presented in a stacked bar graph. Times to IRB approval and activation are presented with Kaplan-Meier methods by Bayesian vs non-Bayesian methods, overall and for the most common phases.

Results

Between January 2009 and December 2013 we identified 1020 trials meeting our selection criteria, of which 283 (28%) included at least one planned Bayesian method. Table 1 presents the protocol features and whether Bayesian methods were included. Between 2009 and 2013, 184 to 219 protocols were submitted each year and the annual fluctuations of Bayesian protocols ranged from 26% to 30%, compared to a smaller number and higher fluctuation between 9% and 29% reported by Biswas, et al. for 2000–2005²². One-third of trials were MD Anderson only (33%) and two-thirds (66%) were multi-centered trials. Trials were primarily industry sponsored (57%), followed by MD Anderson investigator initiated (27%), national cooperative group (9%), or externally peer-reviewed (6%). Only 75 of the 283 (32%) trials with any Bayesian methods were classified as having a Bayesian primary design (i.e., the primary endpoint analysis and/or study design were based on Bayesian calculations and methods). The most common Bayesian designs were adaptive logistic regression modeling (N=22), continual reassessment method (N=20), adaptive randomization (N=16), and toxicity probability interval (N=12). The most common non-Bayesian primary designs are fixed equal randomization (N=269), 3+3 phase I dose escalation (N=206), and single arm trials looking at efficacy (N=132) or toxicity (N=102). All trials having a Bayesian component, but not a Bayesian primary design, used Bayesian methods for interim analyses throughout the trial. They may also have used Bayesian methods for analysis of some secondary or exploratory endpoints. Among such trials, the highest use of Bayesian methods are found in those designed as accelerated titration (50%), single arm efficacy (44%), single arm toxicity (30%), or feasibility studies (29%). Regarding primary endpoints, 41% of the trials studied toxicity while 21% of the trial studied response. Progression-, recurrence-, disease- , or event-free survival were studied in 18% of trials, while 7% of trials studied overall survival. We further classified the primary endpoint based on the data type. The most common type in our clinical trials are binary endpoints (70%), such as response or presence of defined toxicities, and time-to-event outcomes (23%), such as overall or recurrence-free survival. Only 5% of the trials had a continuous primary endpoint. Nearly half of the trials had no control (48%). Of those with a control, an active control arm was the most common (24%), followed by historical (14%) or placebo (13%) control. Most trials had one treatment arm (60%), 34% had two to four arms, and 1% had five or more arms. Interim analyses were planned in 73% of trials, 62% of non-Bayesian and 100% of the Bayesian trials, with 62% or 10% of trials having at least two interim analyses or one interim analysis, respectively. Among specific common disease sites, the highest numbers of protocols submitted were for blood (hematologic) malignancies (N=398; 39%), followed by other cancers (11%) and all types of advanced cancers (10%). For specific disease sites, gastrointestinal (8%), skin (7%), breast (7%), lung (6%), genitourinary (6%), gynecological (6%), and brain (5%) cancers were more common. Protocols for genitourinary (N=26, 43%), skin (N=24, 34%), hematologic (N=127, 32%), and gastrointestinal cancers (N=25, 30%) had the highest use of Bayesian trials among disease sites.

Table 1.

Protocol Features by Bayesian Status

	Bayesian
Protocol Feature	Yes N (%)a	No N (%)a	All N (%)b
All	283 (28%)	737 (72%)	1020 (100%)
Year
2009	57 (27%)	155 (73%)	212 (21%)
2010	57 (26%)	162 (74%)	219 (21%)
2011	57 (28%)	146 (72%)	203 (20%)
2012	52 (28%)	132 (72%)	184 (18%)
2013	60 (30%)	142 (70%)	202 (20%)
Site
MD Anderson Only	189 (56%)	146 (44%)	335 (33%)
Multi Center	94 (14%)	582 (86%)	676 (66%)
Other Sites Onlyc	0 (0%)	9 (100%)	9 (1%)
Study Source
Externally Peer-Reviewed	12 (19%)	52 (81%)	64 (6%)
Industry	112 (19%)	469 (81%)	581 (57%)
Institutional	153 (56%)	120 (44%)	273 (27%)
National Cooperative Group	6 (6%)	89 (94%)	95 (9%)
Missingd	0 (0%)	7 (100%)	7 (1%)
Primary Design
Bayesian Primary Design
Adaptive Bayesian Logistic Regression Model	22 (100%)	0	22 (2%)
Bayesian Adaptive for Efficacy and Toxicity	4 (100%)	0	4 (0%)
Bayesian Adaptive Phase II Design	1 (100%)	0	1 (0%)
Continual Reassessment Method	20 (100%)	0	20 (2%)
Randomization Adaptive	16 (100%)	0	16 (2%)
Toxicity Probability Interval	12 (100%)	0	12 (1%)
Bayesian Primary Design Subtotal	75 (100%)	0	75 (7%)
Non-Bayesian Primary Design
3+3	37 (18%)	169 (82%)	206 (20%)
Accelerated Titration	4 (50%)	4 (50%)	8 (1%)
Assess Accuracy vs Gold	1 (25%)	3 (75%)	4 (0%)
Standard
Cross-over	0	1 (100%)	1 (0%)
Factorial Design	0	5 (100%)	5 (0%)
Feasibility Study	5 (29%)	12 (71%)	17 (2%)
Non-randomized Multiple Arms	2 (10%)	19 (90%)	21 (2%)
Randomization Fixed Equal	32 (12%)	237 (88%)	269 (26%)
Randomization Fixed Unequal	6 (10%)	52 (90%)	58 (6%)
Randomization Pocock-Simon	1 (33%)	2 (67%)	3 (0%)
Single Arm 2-stage	24 (28%)	62 (72%)	86 (8%)
Single Arm Efficacy	58 (44%)	74 (56%)	132 (13%)
Single Arm Toxicity	31 (30%)	71 (70%)	102 (10%)
Single Arm Other	7 (21%)	26 (79%)	33 (3%)
Non-Bayesian Primary Design Subtotal	208 (22%)	737 (78%)	945 (93%)
Primary Endpoint Feature
Biology	6 (17%)	30 (83%)	36 (4%)
Feasibility	5 (21%)	19 (79%)	24 (2%)
Image	0	3 (100%)	3 (0%)
Overall Survival	2 (3%)	66 (97%)	68 (7%)
Progression- / Recurrence- /Disease- /Event-free Survival	36 (20%)	143 (80%)	179 (18%)
Pharmocokinetics	0	6 (100%)	6 (1%)
Quality of Life	1 (33%)	2 (67%)	3 (0%)
Response	76 (35%)	142 (65%)	218 (21%)
Smoking Abstinence	0	3 (100%)	3 (0%)
Symptom Control	11 (24%)	34 (76%)	45 (4%)
Toxicity	139 (33%)	280 (67%)	419 (41%)
Other	7 (44%)	9 (56%)	16 (2%)
Primary Endpoint Type
Binary	233 (32%)	485 (68%)	718 (70%)
Categorical	1 (50%)	1 (50%)	2 (0%)
Ordinal	4 (29%)	10 (71%)	14 (1%)
Continuous	10 (21%)	37 (79%)	47 (5%)
Time to Event	35 (15%)	203 (85%)	238 (23%)
None/Compassionate Use	0	1 (100%)	1 (0%)
Control
Active Control	45 (18%)	200 (82%)	245 (24%)
Historical Control	46 (32%)	96 (68%)	142 (14%)
Placebo Control	16 (12%)	119 (88%)	135 (13%)
Self-Control	1 (17%)	5 (83%)	6 (1%)
None	175 (36%)	317 (64%)	492 (48%)
Maximum Number of Arms
1	214 (35%)	396 (65%)	610 (60%)
2−4	64 (16%)	338 (84%)	402 (34%)
>5	5 (63%)	3 (38%)	8 (1%)
Number of Interim Analysese
0	0	280 (100%)	280 (27%)
1	0	105 (100%)	105 (10%)
>2	283 (45%)	352 (55%)	635 (62%)
Disease Sitef
Advanced	27 (25%)	79 (75%)	106 (10%)
Blood	127 (32%)	271 (68%)	398 (39%)
Brain	14 (28%)	36 (72%)	50 (5%)
Breast	17 (25%)	52 (75%)	69 (7%)
Gastrointestinal	25 (30%)	59 (70%)	84 (8%)
Genitourinary	26 (43%)	35 (57%)	61 (6%)
Gynecological	10 (16%)	52 (84%)	62 (6%)
Head & Neck	6 (19%)	25 (81%)	31 (3%)
Lung	12 (19%)	51 (81%)	63 (6%)
Skin	24 (34%)	46 (66%)	70 (7%)
Other Cancers	26 (23%)	88 (77%)	114 (11%)
Other Non-cancer	5 (42%)	7 (58%)	12 (1%)

^aRow percentages are presented.

^bColumn percentages are presented.

^cThese trials run by MD Anderson investigators with accrual solely at other sites were primarily community setting prevention trials.

^dInformation is missing from the source database.

^eMany analyses have an unspecified number with ongoing continuous monitoring. These protocols were included as ≥2.

^fSites will sum to more than 100% because many protocols include multiple sites.

Figure 1 shows that the most common type of trials were phase II (N=351), followed by phase I (N=260), phase I-II (N=176), and phase III (N=166). Bayesian methods were commonly used in early phase trials, with their applications in 47% of phase I/II, 34% in phase II, and 24% of phase I trials. Supplemental Table 1 presents the design and endpoint features by phase of the trial. The Bayesian logistic regression model and continual reassessment method were the most common Bayesian methods implemented in phases I or I/II, and adaptive randomization was the most common in phases II or III. Among trials with non-Bayesian primary designs, the 3+3 dose escalation was the most common in phases I or I/II while fixed equal randomization for phase II or III.

Figure 1.

Number of protocols and percent of Bayesian protocols by phase of development. The shaded black portion presents the number of protocols using at least one Bayesian method and the percentage shows the percentage of trials that are Bayesian within each phase category. The number at the top of the bar provides the total number of protocols submitted for each phase. There were no (0%) phase IV Bayesian trials submitted.

The numbers and percentages of Bayesian trials for each combination of study site (MD Anderson vs multi-center) and funding source (industry vs other) are presented in Table 2. MD Anderson-only trials had 56% to 57% of trials that had Bayesian components regardless of the funding source up from 23% to 30% in our previous report. Multi-center trials had 14% Bayesian trials, regardless of funding source, which is also up from 5% to 8% previously. Overall, 56% of the MD Anderson-only trials and 14% of the multi-center trials applied Bayesian methods. However, due to the fact that the vast majority of trials among the industry funding were multi-center trials (N=509, 88%), only 19% of the industry funding trials applied Bayesian methods. On the other hand, a majority of the other funding trials were MD Anderson-only trials (N=263, 61%), 40% of which involved Bayesian methods. Another notable change is that MD Anderson-only trials were much more likely to be industry-sponsored (72/335 [21%]) compared to the previously report (13/570 [2%]). Overall, the total percentage of trials that applied Bayesian methods only increased slightly, growing from 20% previously to 28% in the current report.

Table 2.

Industry-sponsored and other clinical trials classified by site.^a

Site	Industry Funding	Other Funding	Totalb
MD Anderson Only	41/72 (57%)	148/263 (56%)	189/335 (56%)
Multicenter	71/509 (14%)	23/169 (14%)	94/678 (14%)
Total	112/581 (19%)	171/432 (40%)	283/1013 (28%)

^aNumbers of Bayesian Trials/total numbers of trials are shown along with the percentage of Bayesian trials indicated in parenthesis.

^bThe total numbers of trials are smaller than 1020 due to 7 multicenter trials with missing funding information.

The specific Bayesian design details are presented in Table 3. Among 283 Bayesian trials, five had three Bayesian methods, 86 had two, and 192 had one Bayesian method specified. Only 41 (14%) of the trials used informative priors. The most common methods were toxicity monitoring (65%), efficacy monitoring (36%), and dose finding (22%). The percentages of different Bayesian methods add up to more than 100%, because 91 protocols had multiple methods. For trial conduct and interim analyses, the most common way to carry out the Bayesian method was to use pre-calculated boundaries included in the protocol (N=143, 51%). This was followed by 17% requiring a statisticians’ analysis throughout the study, 13% using the Clinical Trial Conduct site with real-time analyses of data maintained by the clinical staff throughout the trial, while the remaining 19% used other means, such as trial-specific software or not specified.

Table 3.

Details of Bayesian Studies

Bayesian Characteristics	N (%)
All	283 (100%)
Number of Bayesian Methods
1	192 (68%)
2	86 (30%)
3	5 (2%)
Informative Prior
No	242 (86%)
Yes	41 (14%)
Methoda
Toxicity Monitoring	183 (65%)
Efficacy Monitoring	103 (36%)
Dose Finding	61 (22%)
Adaptive Randomization	16 (6%)
Predictive Probability	11 (4%)
Hierarchical Model	5 (2%)
Conduct
Clinical Trial Conduct Website	38 (13%)
Pre-calculated Boundaries	143 (51%)
Statistician Throughout the Study	47 (17%)
Unspecified/Other	55 (19%)

^aThese frequencies will add to more than 100% because 91 protocols had multiple methods.

Figure 2 presents the planned sample sizes for early phase trials and shows that, regardless of phase, there is a trend for slightly smaller sample sizes among Bayesian trials. This is most notable in the third quartiles of phase I-II and phase II trials and the medians of phase II trials. This may be in part due to multicenter trials, which tend to be larger and non-Bayesian. This may also be in part due to the ability of Bayesian probabilities to be calculated without adjustment for type I error, allowing for smaller planned sizes when multiple interim analyses are planned. For presentation, the graphs were truncated at 500 patients. There were six trials with sample sizes over 500, one non-Bayesian phase I, one Bayesian phase I/II, and two Bayesian and two non-Bayesian phase II trials.

Figure 2.

Sample sizes for Bayesian and non-Bayesian trials for phase I, I/II, and II trials. For presentation, the graphs were truncated at 500 patients. There were 6 trials with sample sizes over 500: 1 non-Bayesian in phase I; 1 Bayesian in phase I/II; and in phase II there were 2 each in Bayesian and non-Bayesian trials.

One question about Bayesian trials is whether it is harder to get a positive review and move to activation. Table 4 presents the protocol status and time to key approvals and activation. The first two rows show the MD Anderson planned and actual accruals. Note that the full protocol accrual information was not available from the institutional protocol database. Since MD Anderson-only trials are more likely to be Bayesian, the Bayesian trials in this table are more likely to represent the full protocol accrual numbers. Many of the non-Bayesian trials are multicenter protocols, where MD Anderson will accrue a small portion of the patients. The median time from submission to IRB approval was 1.4 months for all trials, with or without Bayesian methods, regardless of protocol phase (Supplemental Figure 1). Bayesian trials did take slightly more time for activation with 6.9 months vs 6.2 months for trials without Bayesian methods. Later phase trials trended towards longer activation times. The differences between Bayesian and non-Bayesian trials is only apparent in Phase I and I-II trials (Supplemental Figure 2). All but 37 trials (16 Bayesian, 21 non-Bayesian) were activated as of October 1, 2014. A supplemental review of these 37 trials at the time of manuscript revision showed that 31 were activated (15 Bayesian, 16 non-Bayesian) and 6 were withdrawn or terminated without activation (1 Bayesian, 5 non-Bayesian).

Table 4.

Protocol Status

	Bayesian
Protocol Status	Yes	No	All
All	283 (100%)	737 (100%)	1020 (100%)
MD Anderson Accrual Planned - N=1020	42.0	24.0	30.0
Median (Interquartile Range)	(25.0, 72.0)	(15.0,50.0)	(15.0,60.0)
[minimum, maximum]	[2.0, 350]	[0.0, 1280]	[0.0, 1280]
MD Anderson Accrual Actual - N=1005	17.0	8.0	11.0
Median (Interquartile Range)	(6.0, 39.0)	(3.0, 22.0)	(3.0, 27.0)
[minimum, maximum]	[0.0, 334]	[0.0, 662]	[0.0, 662]
Percent MD Anderson Accrual Actual/Planned - N=1000	46.7%	40.0%	40.0%
Median (Interquartile Range)	(18.7%, 75.0%)	(15.0%, 70.0%)	(15.4%, 70.2%)
[minimum, maximum]a	[0.0%, 100%]	[0.0%, 100%]	[0.0%, 100%]
Months to IRB Approval - Events/N median (95% Confidence Interval)	283/283	737/737	1020/1020
	1.4 (1.2, 1.4)	1.4 (1.2, 1.4)	1.4 (1.2, 1.4)
Months to Activation - Events/N median (95% Confidence Interval)	267/283	716/737	983/1020
	6.9 (6.1,7.7)	6.2 (5.8, 6.6)	6.4 (6.0,6.7)
Protocol Status - N (%)
Waiting for Activation	7 (2%)	8 (1%)	15 (1%)
Ongoing	135 (48%)	292 (40%)	427 (42%)
Closed to New Patient Enrollmentb	110 (39%)	407 (55%)	517 (51%)
Completed	31 (11%)	30 (4%)	61 (6%)

^aSome trials exceeded MD Anderson planned accrual. For these trials, values were truncated at 100% accrual. This was primarily due to MD Anderson investigators getting permission to enroll more patients on multicenter trials than originally planned.

^bThis may be temporary for multiple administrative reasons or final and waiting for patients to complete the study.

Case studies for the impact of Bayesian clinical trials.

In this section, we focus on the clinical impact of two case studies of multi-arm adaptive trials.

BATTLE trials

The BATTLE-1 trial was the first completed, prospective biopsy-mandated, biomarker-based, Bayesian adaptive randomized clinical trial for patients with advanced non-small cell lung cancer. The trial involved four targeted therapies with eleven molecular biomarkers to (1) test the treatment efficacy of the targeted agents using disease control rate; (2) identify the corresponding prognostic and predictive markers; and (3) treat patients with the most effective treatment in the study based on the available data. Adaptive randomization was applied to assign more patients to better treatments based on the individual patient’s biomarker profile. A Bayesian hierarchical probit model was constructed to borrow information across biomarker groups within the same treatment to improve the efficiency of treatment effect estimation.²³ Results have been reported elsewherevs.²⁴

The BATTLE-2 trial used a Bayesian logistic regression model. Potential prognostic and/or predictive biomarkers were identified during the training phase (pre-BATTLE-2) on the basis of prior studies and the literature and then tested and validated in two stages.²⁶ Results are published elsewhere.²⁵ The BATTLE program demonstrated the feasibility and promise of novel biomarker-based clinical trial platforms, moving clinical research one step closer to personalized medicine.

I-SPY 2 TRIAL

The I-SPY 2 TRIAL^27–29 is a multicenter, adaptive, open-label, phase II, neo-adjuvant, platform³⁰ trial identifying potential new treatments for patients with a high risk of breast cancer recurrence.^{10, 31} Patients are assessed for hormone receptor (HR) status, HER2 status, and a high risk genetic profile by the MammaPrint Assay.³² The available experimental treatments change as treatments become available or are selected to move on to a phase III trial (graduate), as described previously.^{28, 29} A treatment graduates when the predictive probability of success in a phase III trial against the standard treatment is greater than 0.85, as calculated throughout the trial.²⁹

To date, this trial has resulted in two treatment combinations have been reported^27,29 that show sufficient promise to warrant a phase III trial.

Another benefit of this trial design is the coming together of researchers and pharmaceutical companies dedicated to a collaborative approach to fighting cancer. By pooling resources to answer multiple questions more efficiently, the nature of research has been transformed.²⁷ The flexibility of Bayesian trial design enables this complex design to adapt to incoming information in a pre-specified, organized way, maximizing patient and institution resources over many years.

Conclusions

Our Bayesian trial experience has remained relatively constant over 5 years (26%–30%), whereas it fluctuated in the original report by Biswas, et al²² (9%–29%). It seemed that many more trials were using Bayesian designs, so we took a closer look at Table 1 from the original report²² and our Table 2. We found a partial Simpson’s paradox.³³ In every category, the percentages of Bayesian trials increased by 75% to 180%, but the overall percentage of Bayesian trials only increased by 40% (20% vs 28%). This increase is most noticeable in MD Anderson-only trials. If funding came from pharmaceutical vs other funds, 57% vs 56% of trials were Bayesian compared to only 23% vs 30% in the original report. The small increase in overall percentage is due to the notable shift to more multicenter trials as inter-institutional collaboration is increasingly encouraged and supported. This is combined with a large shift to pharmaceutical funding, as governmental funding reduced. However, the 71 multi-center Bayesian trials initiated with industry sponsors mark an important increase over the 7 Bayesian trials initiated previously. Based on that prior number, Gönen noted that that the design of Bayesian trials was confined to MD Anderson’s zip code.²⁰ Beyond this evidence of increased Bayesian methods use in our data, Bayesian trial use also is expanding to multiple diseases with international implications. Recent publications present results from Bayesian-designed trials including newborn hypoxic-ischemic encephalopathy,^{34, 35} neurological emergencies,³⁶ atrial fibrillation,³⁷ Alzheimer’s,³⁸ and Ebola³⁹ while expanding to evaluate multiple treatments or diseases at once.^40–43 The NIH and FDA supported the development of Bayesian adaptive trials⁴⁴ with notable activity and mixed enthusiasm.^{36, 45–48} Recently, the FDA has enhanced its capacity to review complex innovative designs to further advance the benefits of implementing such trials.⁴⁹

Bayesian and frequentist methods provide complementary views of statistical inference.⁵⁰ Many Bayesian and frequentist adaptive designs, as well as several hybrid designs, have been proposed to increase the flexibility of trial conduct and improve the efficiency of clinical trials by finding the optimal sample sizes.^51–54 Not all Bayesian methods are ideal or necessary in every scenario of clinical trials, and often there are reasonable frequentist counterparts. However, there are several areas where Bayesian methods are producing a positive impact on clinical trial aspects that can ultimately speed cancer therapy discovery. Each year there are new methods proposed that provide good statistical operating characteristics in dose escalation trials,⁵⁵ for example the Bayesian optimal interval design^{56, 57} and keyboard design.⁵⁸ As the cutoff for this survey was December 2013, the increasing use of these novel designs is not reflected here. For example, the Bayesian optimal interval has been the most downloaded Bayesian design (1094 downloads) via the MD Anderson Software Download site from January 2016 to January 2019. Such novel designs provide a replacement to the 3+3 escalation practice that many physicians have used as a default for decades. As new Bayesian methods are developed that also allow the dose adjustment rules to be printed directly in the protocol,^59–61 their implementation will improve the efficiency of drug discovery. Research teams can arrive at the targeted dose more accurately with easy steps for the medical teams to implement. We saw an increase in Bayesian phase I trials from 13% to 24% between the two reporting periods. Additionally, 22% of current Bayesian trials implement dose finding methods compared to 9% previously. The development of novel Bayesian phase I-II designs further streamlines and accelerates drug discovery.⁶² Another benefit of Bayesian trials is that, when informative priors are properly applied, they can be planned with fewer patients in the same phase group,⁶³ preserving patients for other research questions. Bayesian trials allow us to borrow information across subtypes while investigating differences among them.^64–66 With the increasing size of clinical trials, interim analyses become more important, and is a natural role for Bayesian designs.⁶⁷ Note that frequentist’s group sequential designs have been well-established and widely applied for multiple testing while controlling the type I error.⁶⁸ Meanwhile, Bayesian methods provide the same desirable features, but they are more flexible and simple to interpret.⁶⁹ One protocol feature that has changed between the two reporting periods is the increase of formal toxicity monitoring in trials (27% previously to 65% currently). This may be due to an institutional policy that phase II trials using treatments with limited prior safety data must have formal toxicity monitoring. We commonly implement this with sequential methods¹⁴ printed in the protocol.

Bayesian designs have many strengths, but this does not mean they provide solutions to all problems. Some clinical investigators have the misconception to implement Bayesian designs with very small sample sizes, assuming that any size is ok as long as we’re doing a Bayesian trial. To address this, the operating characteristics still need to be carefully evaluated, and the importance of robust and desirable performance needs to be included in protocol planning. Also, when historical information is being included as part of an informative prior, the relevance and similarity of that information from those patients needs to be carefully considered. The communication with the investigator needed to elicit a meaningful and appropriate prior takes deliberate time, effort, and skill.²¹ Operating characteristics from several priors need to be evaluated to provide knowledge about the impact of prior selection on the conclusions. Additionally, to work appropriately, Bayesian methods often rely on similar statistical assumptions as frequentist approaches. For example, when using Bayesian adaptive randomization, the assumption of constancy of time-trend needs to be acknowledged and examined at the planning and analysis stages to accommodate the potential of population drift over time.^70–72 Without careful planning, adaptive randomization, Bayesian or frequentist, can result in reduced power and a risk of more patients being assigned to the less effective arm, compared to equal randomization. Measures can be taken, such as starting with an equal randomization phase then switch to adaptive randomization and restricting the randomization probability to 0.1 to 0.9, to avoid extreme allocation and undesirable properties.^73–75

There is still need for new trial methods. As cancer therapy continues to change, some treatments are considered useful if they turn a deadly cancer into a chronic disease. Longer term outcomes, such as progression-free or overall survival are being implemented in earlier phase trials, since the traditional response either is not expected to be met or does not act as a reasonable surrogate marker for survival. When designing a trial, it is tempting to select a surrogate marker that is quick and easy to measure,²⁷ but the need to ensure that the surrogate marker accurately predicts the long-term goal is growing. This becomes more complicated as personalized medicine allows patients to receive a larger variety of treatments in differing orders. Having easy-to-use methods available to allow sequential monitoring of incomplete and time-to-event endpoints, as well as comparing these endpoints for futility in early randomized trials, will fill an important gap in trial design methods.

In recent years, there is a growing interest in applying master protocols to study multiple therapies, multiple diseases, or both. Bayesian and frequentists methods have been proposed in the settings of umbrella trials, basket trials, enrichment designs, and platform designs.^{43, 76} The critical reviews of such designs in terms of strengths, weaknesses, and current use have been reported.^{77, 78}

Since there often is a long turn-around time from developing a new method to finding it used in medical literature,^{17, 18, 21} more educational efforts are needed to ease the concerns of clinicians who may be hesitant to use methods their colleagues have never seen. Some Bayesian methods for delayed endpoints are already available with software implementation.^79–81 To help disseminate and transfer the knowledge of applying Bayesian methods in clinical trials, we have developed many R-Shiny applications which are available at the MD Anderson Software Online site [https://biostatistics.mdanderson.org/softwareonline/] and at http://trialdesign.org. In addition, more than 80 downloadable programs for clinical trial design and analysis are available at the MD Anderson Software Download site [https://biostatistics.mdanderson.org/softwareDownload/]. To date, we have more than 21,000 downloads from all over the world, since launching the site in 2004. When we proactively implement new methods with motivated medical colleagues and lead by example, we help to advance the adoption of more efficient trials designs, Bayesian or non-Bayesian alike.

New clinical trial designs continue to play an important role in cancer clinical trials. The advancement in targeted and immunotherapies opens the door to new methodological needs⁸². The complexity, flexibility, and learning process inherent in Bayesian methods make this a promising field for advancing the fight against cancer, as well as providing excellent career opportunities for statisticians and clinicians to make a difference in reaching that cure.

Supplementary Material

1

Supplemental Table 1. Protocol Design by Phase

Supplemental Figure 1. Time from submission to IRB approval by Bayesian trials (A) overall and (B) within phase I , (C) phase I-II, (D) phase II, or (E) phase III trials. IRB approval was determined within our data collection period. All included trials were approved as of Oct 1, 2014. Note, the phase I protocol with >12 months' time is censored at 12.2 months for presentation, but was approved after 16.6 months. It was inactive in the review process for 9 months, when the originally proposed agent became FDA approved. During that time, the protocol was revised to include a novel combination. The time from this resubmission to IRB approval was 2.25 months. The phase II Bayesian protocol that took 11.3 months was initially submitted without a statistical collaborator. Multiple statistical and non-statistical revisions were required from all reviewers. The time from statistical review to acceptance of statistical aspects was 3.6 months. The modifications in response to all reviewers took the remainder of the time.

Supplemental Figure 2. Time from submission to activation by Bayesian trials (A) overall and (B) within phase I, (C) phase I-II, (D) phase II, or (E) phase III trials. Activation was determined within our data collection period. Trials that were not activated as of Oct 1, 2014 were censored on Oct 1, 2014. There were three non-Bayesian trials that had activation times greater than three years. All were withdrawn after the data cut off. One was withdrawn because of the difficulty of implementing treatment, and IND approval took nearly 4 years after IRB approval. The other two trials were dormant without activation. They were withdrawn years later, due to department lack of priority for the study.