Bayesian Methods in Human Drug and Biological Products Development in CDER and CBER

Abstract

The Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) of the U.S. Food and Drug Administration (FDA) have been leaders in protecting and promoting the U.S. public health by helping to ensure that safe and effective drugs and biological products are available in the United States for those who need them. The null hypothesis significance testing approach, along with other considerations, is typically used to demonstrate the effectiveness of a drug or biological product. The Bayesian framework presents an alternative approach to demonstrate the effectiveness of a treatment. This article discusses the Bayesian framework for drug and biological product development, highlights key settings in which Bayesian approaches may be appropriate, and provides recent examples of the use of Bayesian approaches within CDER and CBER.

Introduction

Under the Federal Food, Drug, and Cosmetic Act, a drug’s effectiveness must be established by “substantial evidence,” which is “evidence consisting of adequate and well-controlled investigations, including clinical investigations, by experts qualified by scientific training and experience to evaluate the effectiveness of the drug involved, on the basis of which it could fairly and responsibly be concluded by such experts that the drug will have the effect it purports or is represented to have under the conditions of use prescribed, recommended, or suggested in the labeling or proposed labeling thereof.” [1] FDA has also generally considered “substantial evidence” of effectiveness to be necessary to support licensure of a biological product under Section 351 of the Public Health Services Act [2]. The FDA regulatory review determines whether “substantial evidence” has been demonstrated. Additionally, because all drugs have the potential for adverse effects, FDA integrates a structured benefit-risk assessment as part of the regulatory review of marketing applications for drugs and biological products [3]. The strength of evidence in each trial contributing to meeting the substantial evidence standard is assessed by appropriate statistical methods [4].

The Bayesian approach has a wide variety of potential applications in drug and biological product development [5]. However, there is a limited number of submissions featuring Bayesian methods. The limited number of submissions featuring Bayesian methods may be, at least in part, a result of insufficient knowledge of Bayesian approaches [6]. This article discusses the use of the Bayesian framework for drug and biological product development, highlights key settings in which Bayesian approaches may be appropriate, and provides recent examples of the use of Bayesian approaches within the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) at the U.S. Food and Drug Administration (FDA).

Areas with Emerging Bayesian Methods Use

Although the use of Bayesian methods may be well suited in some clinical contexts, the methods may be considered regardless of study type, population, or therapeutic area, as appropriate. As with any proposal for a clinical trial, whether the proposed Bayesian study design and/or analysis is fit-for-purpose will be determined on a case-by-case basis. Some areas of drug or biological product development are highlighted below.

Early Drug/Biological Product Development

The objective of early-phase studies includes generating preliminary safety and efficacy data to help select drug or biological product dose(s) for subsequent development. There is an opportunity to use Bayesian model-based design and analyses to meet this objective. For example, a Bayesian model could potentially be used to estimate the probability of dose-limiting toxicity based on prior toxicity information or initial assumptions. The estimates could be continuously updated as more data are collected.

Bayesian modeling in early-phase studies has been studied extensively, e.g., [7]. Some of the challenges in early-phase studies using Bayesian methods include clear characterization of the assumptions, study design, and model settings. Generally, providing sufficient details to describe and justify Bayesian study design, statistical model(s), and assumptions in proposals for early-phase studies is helpful to gain alignment among stakeholders. Detailed description and justification of the optimal and efficient designs (if applicable), sample size, duration of toxicity monitoring, potential dose levels, the model, prior distribution and its informativeness, parametrizations, and success criteria are especially helpful. Simulation results with operating characteristics under various scenarios and assumptions provide valuable information to assess a proposal for an early-phase study [8].

Noninferiority

Bayesian methods are not limited to superiority trials and can be used in noninferiority (NI) trials as well. The goal of an NI trial is often to show that the difference between a new investigational product and an active control is small enough to allow the active control’s known effectiveness to support the conclusion that the new investigational product is also effective. An NI study seeks to show that the amount by which the investigational product is inferior to the active control is less than some prespecified NI margin. The FDA guidance on noninferiority trials acknowledges the potential utility of a Bayesian approach in this setting [9]. As examples, Bayesian methods can potentially be used in either the design of an NI trial (e.g., to determine the NI margin) or the analysis of the data generated by the trial. Use of Bayesian approaches can be particularly well suited to determining the NI margin, which involves synthesizing data from past studies. Some examples of the use of Bayesian methods to determine the NI margin and to analyze NI trial data can be found in Rothwell et al. [10], Gamalo et al. [11], and Price and Scott [12].

Adaptive Clinical Trials

Some considerations on Bayesian adaptive study designs are included in the guidance for industry, Adaptive Design Clinical Trials for Drugs and Biologics. An adaptive design allows for prospectively planned modifications to the study design based on accumulating data from participants in the clinical trial. Bayesian adaptive study designs use the Bayesian framework for study adaptations and/or posterior distributions for decision-making. Examples of Bayesian adaptive design features include the following categories per Adaptive Design Clinical Trials for Drugs and Biologics:

• “Use of predictive statistical modeling, possibly incorporating information external to a trial, to govern the timing and decision rules for interim analyses.

• Use of assumed dose–response relationships to govern dose escalation and selection.

• Explicit borrowing of information from external sources (e.g., previous trials, natural history studies, and registries) via informative prior distributions to improve the efficiency of a trial.

• Use of posterior probability distributions to form criteria for trial success.”

Pediatric Clinical Trials

Informative Bayesian methods can be a good fit for pediatric studies, as these methods can allow the incorporation of prior information about efficacy from adults or other source populations (for example, using information from adolescents in drawing inferences on younger children), when clinically appropriate, to facilitate a trial design that requires the enrollment of fewer children [13]. Bayesian studies are consistent with the established concept of pediatric extrapolation, which allows for efficacy to be assessed in pediatric patients with support from information gathered in other populations [14]. Extrapolation, when scientifically justified, is particularly attractive in pediatric investigations as it can allow us to reduce the size or need for studies in children who are a vulnerable population in need of extra patient protections. Bayesian methods accomplish this by creating a prior distribution based on the information leveraged from the source population which can be used in the analysis in the target pediatric population.

In the past, extrapolation was employed using a decision-tree approach, through which the development program could be reduced if certain thresholds of evidence were reached [15]. Bayesian methods allow more flexibility for pediatric studies, given that the prior information can be weighted based on the level of applicability rather than an all-or-nothing approach to incorporating the information. Deciding upon applicability a priori is preferable because it separates applicability from outcomes of the new study, which may bias reviewers with regard to how previous studies are viewed.

These methods were discussed in a September 2021 joint FDA and Maryland Center for Excellence in Regulatory Science and Innovation workshop, “Advancing the Development of Pediatric Therapeutics Complex Innovative Trial Design Public Workshop,” which included presentations and discussion on the use of Bayesian methods in pediatric clinical trials from FDA, the European Medicines Agency, and industry [16]. There are also ongoing international efforts within the International Council for Harmonization to gain alignment on the scope and applicable methodologies of pediatric extrapolation generally, including the use of Bayesian methods [17].

First, to do extrapolation, one needs to create the prior distribution containing the borrowed information intended for use in the analysis. FDA has seen several different proposed approaches, but they tend to follow a similar process:

1. Determine the relevant data to be used in the construction of the prior distribution.

2. Synthesize the different sources of information. This involves weighing the relative relevance of the different sources to the new scientific question under investigation.

3. Determine the final overall weight of the prior based on balancing the amount of prior evidence, the uncertainty about the applicability of this evidence, and the required sample size for the study.

The main challenges involved in constructing the prior distribution are synthesizing the information and determining the level of weighting. Synthesizing the information becomes complex when a range of heterogeneous data sources are used. For example, if we have a mix of adult and pediatric clinical trials, or if some of the trials used different designs than the study being planned while others used the same design, then we have different amounts of relevance. More complex statistical models may be used to reflect these relevant imbalances. In all cases, there are necessary subjective judgments to be made in building these models, requiring multidisciplinary collaboration in order to design and implement. Some examples of these subjective judgments include assessments about the degree of heterogeneity between trials and the relative relevance of the different sources of information. Schmidli et al. [18] provides an example of this process in practice, where the authors used information from both adult and pediatric studies for fingolimod to create a prior distribution for a study utilizing fingolimod as an active control in pediatric patients.

Discussions related to how to weight the prior information relative to the information collected in the trial can be challenging as finding an acceptable balance in weighting between the information collected in the trial and the prior information is critical in order for the results of the subsequent analyses to be persuasive. If the prior information is weighted too heavily, then the prior information will overpower the data generated from the pediatric population, and the result may not be persuasive. If too little weight is given to the prior information, the pediatric trials may need to enroll more pediatric patients than might be necessary.

Rare Diseases

Similar to pediatric clinical trials, rare disease clinical trials present challenges that may also offer an opportunity for the Bayesian framework. These methods can be useful when there is a limited pool of patients, and data exist that can inform a prior distribution, such as data from earlier clinical trials. Registry data or other real-world data might also be useful for informing a prior distribution, if such use is sufficiently justified in terms of data quality, applicability, and other aspects [19, 20]. In such cases, evidence from various sources can be integrated into the assessment with uncertainty expressed in a probability scale. Interpretation from the functions of the posterior probability distribution can often be straightforward if the early study design strategy is appropriately deliberated and selected [21].

Although designing rare disease clinical trials is different from designing pediatric clinical trials and presents its own distinct set of issues, the approaches for choosing a prior distribution as discussed for pediatric clinical trials would also apply for rare disease clinical trials. What constitutes an appropriate prior distribution for a rare disease will vary depending on the experimental treatment, control treatment, study population, and the degree of similarity between prior and prospective data. It is useful to incorporate a skeptical prior, which quantifies skepticism that the treatment is beneficial or harmful. As the therapeutic landscape evolves, the relevance of prior data for the population of interest may change. Assessing data quality attributes (e.g., source, usability, timing, accuracy, completeness) and comparability attributes (e.g., demographics, disease features, phenotype, genotype, disease progress, standards of care) are helpful to ensure that analysis results using prior data will provide usable and reliable information that is fit for regulatory purposes.

Subgroup Assessment

A subgroup is a given subset of a clinical trial population that has a certain characteristic(s). Along with the evaluation of a treatment effect in the overall study population, assessment within subgroups of patients where the efficacy may vary is important.

Traditionally, treatment effects in an individual subgroup in a pivotal trial are estimated by the observed effect within that subgroup. Although these direct estimates are easily understood, the observed treatment effects can be highly imprecise estimates of the true treatment effects. The variability of observed treatment effects across the subgroups in a trial is generally greater than the variability of the true subgroup treatment effects [22]. The observed variability of treatment effects across subgroups can be extreme when subgroups are small, given that observed treatment effects are more susceptible to random highs and lows of the sample data. Bayesian methods that weigh data in different subgroups in a way that increases the precision of the subgroup treatment effect estimates can correct these extremes by pulling the estimates toward the overall treatment effect in the total study population.

Bayesian hierarchical models for subgroup analysis have been discussed extensively in the literature and featured in the CDER impact stories as an innovative statistical approach to provide the most reliable treatment outcomes information to patients and clinicians [23]. The main idea is to analyze relevant subgroups together rather than in isolation. The Bayesian method accomplishes this by linking individual subgroups through a prior distribution encompassing underlying treatment effects in each subgroup. Borrowing information from other subgroups improves the overall accuracy of the estimates across the subgroups. This property is particularly useful for the analysis of data from the smaller subgroups.

Examples of Bayesian Methods Use in CDER- and CBER-Reviewed Clinical Trials

Example 1: Pediatric Trials (Approval, Belimumab) [24]

In 2018, FDA received a New Drug Application (NDA) supplement containing the results of a Pediatric Research Equity Act (PREA)-required postmarketing study for belimumab, a B-lymphocyte stimulator (BLyS)-specific inhibitor, in the treatment of pediatric patients with active, autoantibody-positive systemic lupus erythematosus (SLE) who were receiving standard therapy. Due to the rarity of the disease in children, an adequately powered pediatric study was not feasible but a study nonetheless ran for over 5 years and managed to enroll and randomize 93 participants.

The primary endpoint was the proportion of patients who met the SLE Responder Index (SRI) Response criteria at Week 52. The results of the primary analysis for this endpoint are shown in Table 1. The primary analysis failed to demonstrate a statistically significant difference between belimumab and placebo.

Table 1.

Primary efficacy analysis of SRI response rate at week 52 from the pediatric study

	Placebo N = 40a	Belimumab 10 mg/kg N = 53
Response, % (n)	44% (17)	53% (28)
Observed difference	–	9.2%
Odds ratio (95% CI)	–	1.5 (0.6, 3.5)

^aOne subject in placebo did not have a baseline SELENA SLEDAI assessment and, therefore, did not contribute to SRI analyses

Given concerns regarding the feasibility of conducting another study in this pediatric population, the review team considered whether a Bayesian analysis could be informative, incorporating the data collected in the adult SLE studies.

To conduct such an analysis, a prior distribution had to be determined. The review team considered this in a similar fashion to the three steps discussed previously in Section d. First, they identified the relevant data that could be used; specifically, there were two adult efficacy studies that compared two dose levels (1 mg/kg and 10 mg/kg) against placebo. These data were used since the clinical review team believed the disease and patient response to treatment were likely to be similar between the adult and pediatric subjects, given that the pediatric and adult diseases have similar underlying pathophysiology and management, with BLyS, the target of belimumab, being similarly relevant; systemic belimumab exposures were also similar. The data from the 1 mg/kg dose arms were not considered relevant because the pediatric study only used the higher dose. The results from the adult studies for the 10 mg/kg dose are shown in Table 2.

Table 2.

Primary efficacy analysis of SRI response rate at week 52 from the adult studies

	Adult study 1		Adult study 2
	Placebo N = 275	Belimumab 10 mg/kg N = 273	Placebo N = 287	Belimumab 10 mg/kg N = 290
Response, % (n)	34% (93)	43% (118)	44% (125)	58% (167)
Observed difference	–	9%	–	14%
Odds ratio (95% CI)	–	1.5 (1.1, 2.1)	–	1.8 (1.3, 2.6)

Next, the data from the adult studies were combined to produce a single probability distribution.

The final step of the process was to use a mixture prior approach to reweight the results to ensure that the adult data did not overwhelm the pediatric data in the analysis. This approach was used to vary the amount of information borrowing between no borrowing (represented by a weight of zero of 0) and full borrowing (represented by a weight of 1) where the adult and pediatric data are essentially pooled together, with every patient counted equally.

The Bayesian analysis was performed for the entire range of weights, from no borrowing to full borrowing, to allow a complete view of the spectrum of outcomes. Point estimates (posterior means) and uncertainty intervals (95% credible intervals) were computed for each weight value and are presented in Fig. 1. As more weight was placed on the adult study results, the point estimates moved toward the overall average slightly and the width of the uncertainty intervals shrank considerably.

Figure 1.

Posterior mean (points) and 95% credibility intervals (lines) of the odds ratio of SRI response.

Source: Reproduced using Table 34 FDA Statistical Review.

Typically, to try to reduce subjective biases, the amount of borrowing should be prespecified. Although it was not prespecified in this case, the clinical team did have a pre-existing belief in the similarity between adults and pediatrics based on the similarity of the disease pathogenesis and management. The Bayesian analysis found that a prior distribution weight of at least 0.55 resulted in posterior probabilities of positive treatment effects of greater than 97.5%. This 0.55 weight was found to be reasonable by the review team, and thus this analysis provided support, along with additional evidence, for the approvability of belimumab in the pediatric SLE population.

Example 2: COVID-19 Vaccine

In response to the COVID-19 pandemic, BioNTech Manufacturing GmBH, in partnership with Pfizer Inc., proposed a phase 1/2 trial of their SARS-CoV-2 mRNA vaccine candidate, BNT162b2. The original protocol for trial C4591001 was finalized on April 15, 2020, and was subsequently amended on July 24, 2020, to include a phase 3 trial. The trial was titled “A phase 1/2/3 Placebo-Controlled, Randomized, Observer-Blind, Dose-Finding Study to Evaluate the Safety, Tolerability, Immunogenicity, and Efficacy of SARS-CoV-2 RNA Vaccine Candidates Against COVID-19 in Healthy Individuals.” Trial C4591001 had a number of different objectives, including objectives related to dose-finding, safety, and immunogenicity. The efficacy analyses in the phase 2/3 portion of the study are most relevant for this discussion. Of note, the results described in this section are based on the data submitted as part of the COMIRNATY original Biologics License Application (BLA); the study was ongoing at that time and there have since been additional analyses.

Over 44,000 subjects, aged 16 years and older, were randomized 1:1 to receive two doses of BNT162b2 or placebo at a 21-day interval. The primary efficacy endpoint was confirmed cases of COVID-19 (see protocol [25] and FDA review documents for exact definition [26]), and the primary efficacy null hypothesis was Vaccine Efficacy (VE) ≤ 30%, where VE = 100 * (1 – incidence rate ratio [IRR]). Under the assumption that the number of cases in each group follows a Poisson distribution, the number of cases in the vaccine group, s₁, has a binomial distribution conditional on the total number of cases, s; s₁ ~ Binomial(s, θ) where $\mathrm{\theta }=\frac{r(1-VE)}{r\left(1,-,V,E\right)+1}$ and r is the ratio of study time between the vaccine and placebo groups. The study was designed with a simple Bayesian efficacy analysis using a beta-binomial model with a weakly informative beta(0.700, 1) prior distribution for θ, chosen to have a prior mean of 0.4118 (corresponding to a VE of 30%). The null hypothesis would be rejected if the posterior probability that VE > 30% exceeded prespecified thresholds at interim or final analyses. The thresholds were chosen to maintain a familywise one-sided Type I error rate of 2.5%. There were also interim futility analyses planned based on the trial’s posterior predictive probability of success, calculated with a beta-binomial model.

At the time of the interim analysis, conducted with a data cutoff date of November 4, 2020, there were four confirmed cases in the BNT162b2 arm and 90 confirmed cases in the placebo arm, yielding a VE of 95.5% with a 95% credible interval of (88.8%, 98.4%). The posterior probability of VE > 30% was 0.9999, which exceeded the prespecified threshold of 0.995. Updated final efficacy analyses were similarly strong [25, 26]. These efficacy findings played a critical role in BNT162b2 becoming the first COVID-19 vaccine authorized in the United States under Emergency Use Authorization (December 11, 2020) and subsequently becoming the first U.S.-licensed COVID vaccine under the trade name COMIRNATY (August 23, 2021), landmark milestones in the pandemic response [27].

Example 3: Drug Trials Snapshots (Approval, Bempedoic Acid)

CDER statisticians have used Bayesian hierarchical models to provide information on treatment effect for various demographic subgroups in the FDA Drug Trials Snapshots program. For approved new molecular entities and original biological products, the Drug Trials Snapshots webpage [28] provides information on study design, results of efficacy, and safety studies, and whether there were differences in efficacy and side effects among various subgroups defined by sex, race, and age.

The subgroup assessment in the Drug Trials Snapshots for bempedoic acid is an example of the use of a Bayesian method [29]. The applicant’s phase 3 clinical development program consisted of two randomized, placebo-controlled trials that enrolled adult patients with heterozygous familial hypercholesterolemia or established atherosclerotic cardiovascular disease who were on maximally tolerated statin therapy but required additional lowering of LDL-cholesterol (LDL-C). A treatment effect was assessed by percent change in LDL-C from baseline to Week 12. At the time of the application review, the difference in the LDL-C change between treatment groups within sexes, age groups, and ethnicities was estimated independently in isolation as well as by borrowing information from other subgroups using a Bayesian hierarchical model.

For the Bayesian analysis of treatment effects in subgroups based on sex, before observing the trial data (i.e., a priori), it was assumed that the underlying treatment effect was not expected to be in a particular order by sex. That is, it was assumed that the probability that the treatment effect in male patients would be greater than in female patients was the same as the probability that the treatment effect in female patients would be greater than in male patients. This exchangeability assumption must hold for the resulting treatment effect estimates to be valid. Furthermore, the treatment effects in male and female subgroups were linked by imposing a prior distribution over the unknown true treatment effects within subgroups. An additional layer of prior distributions, called a hyper-prior, was imposed on the parameters of the prior distribution, so that estimated values of the parameters were driven primarily by the observed data. Mathematical details of the model are included in the FDA’s application review [29]. Estimates for subgroups of age and region were obtained similarly.

The results based on observed data alone and the Bayesian analysis in the same subgroups from one of the phase 3 trials (Trial 047) are presented in Fig. 2. The 95% confidence intervals are shown for the observed sample estimates, and 95% credible intervals are shown for the estimates based on the Bayesian analysis. The credible intervals were derived from the posterior distribution of the average difference in LDL-C for the corresponding subgroups. As expected, 95% credible intervals are mostly narrower for the estimates based on the Bayesian analysis, reflecting the precision gained through borrowing information across all the subgroups. The gain in precision via Bayesian analysis is particularly notable in the subgroups with wider 95% confidence intervals.

Figure 2.

Trial 047 subgroup analyses.

Source: FDA’s review of bempedoic acid tablets (NDA 211,616).

For the bempedoic acid Drug Trials Snapshots, the results from two trials were combined to provide the overall treatment effect estimate for each subgroup [30]. This required an additional layer of hierarchy to link the study-specific treatment effects from the two trials.

Example 4: Complex Innovative Trial Design Clinical Trials

FDA announced the Complex Innovative Trial Design (CID) Pilot Meeting Program in the Federal Register on August 30, 2018 [31]. The goals of the CID pilot meeting program were to

1. Facilitate the advancement and use of complex adaptive, Bayesian, and other novel clinical trial designs and

2. Promote innovation by allowing FDA to publicly discuss the trial designs developed through the pilot meeting program as case studies, including while the drug studied in the trial has not yet been approved by FDA.

The CID pilot meeting program was intended especially for innovative clinical trial designs that require simulations to estimate the operating characteristics. Several examples of CID case studies in the program have been published [12, 32]. Per Federal Register notice on October 20, 2022 [33], FDA is continuing the program as the Complex Innovative Trial Design Paired Meeting Program [32].

Discussion

Fitness-for-purpose of a study design and/or analysis is determined on a case-by-case basis. This article highlighted some settings that might be well suited for Bayesian methods. The Bayesian framework has many benefits as well as challenges [5]. One of the challenges of Bayesian approaches is the absence of a single, universally accepted criterion for study success. Instead, at this time, clinical context (e.g., seriousness of disease, prevalence of disease, unmet medical need) is important in determining an appropriate criterion for study success. However, this flexibility is also a strength of the Bayesian framework because it requires a thoughtful cross-disciplinary discussion about how multiple factors influence both the prior distribution as well as the criteria for study success.

FDA has numerous avenues to facilitate the appropriate use of Bayesian methods in drug and biological product development. Moreover, examples and discussion of the use of Bayesian methods have been highlighted in publications and presentations made by FDA staff e.g., [34–37], in Drug Trials Snapshots, and in several guidance for industry documents [4, 38–40]. FDA is committed to continuing to advance the appropriate use of Bayesian methods through ongoing and future efforts, including publishing a draft guidance on the Use of Bayesian Methodology in Clinical Trials of Drugs and Biologics by the end of the fiscal year 2025 as a part of the Prescription Drug User Fee Act VII commitment [41].

Author Contributions

All authors contributed to the writing and editing of this manuscript.

Funding

No funding was received for this manuscript.

Declarations

Conflict of interest

The authors have no conflicts of interest relevant to this article to disclose.