Streamlining Randomized Clinical Trials for Device Therapies in Heart Failure: Bayesian Borrowing of External Data

Abstract

Background

The Breakthrough Devices Program of the US Food and Drug Administration has accelerated the development and evaluation of medical devices for patients with heart failure. One such device is the Optimizer Smart System, which the US Food and Drug Administration approved in 2019.

Methods and Results

The Optimizer device was evaluated in a pivotal randomized clinical trial (FIX‐HF‐5C [Confirmatory Randomized Trial Evaluating the Optimizer System]) that leveraged Bayesian borrowing of external data to reduce the sample size and determine therapeutic device benefit versus continued medical therapy. Bayesian borrowing is explained in the context of the FIX‐HF‐5C trial, including an overview of the statistical methodologies, regulatory considerations, and interpretations of trial results.

Conclusions

The US Food and Drug Administration Breakthrough Devices Program and novel Bayesian statistical methodology accelerated the path to regulatory approval and patient access to a potentially lifesaving device and may serve as a model for future clinical trials.

Nonstandard Abbreviations and Acronyms

CCM

cardiac contractility modulation

FDA

US Food and Drug Administration

FIX‐HF‐5

Original Randomized Trial Evaluating the Optimizer System

FIX‐HF‐5C

Confirmatory Randomized Trial Evaluating the Optimizer System

NYHA

New York Heart Association

VO₂

Peak oxygen consumption

Clinical Perspective.

What Is New?

• The US Food and Drug Administration approved the Optimizer Smart System in 2019 for patients with heart failure on the basis of a randomized clinical trial that incorporated external data via Bayesian borrowing.

• The manuscript explains Bayesian borrowing in the context of the FIX‐HF‐5C (Confirmatory Randomized Trial Evaluating the Optimizer System) trial, including regulatory considerations and interpretation of results.

What Are the Clinical Implications?

• Novel Bayesian methods accelerated the regulatory process and may serve as a model for future trials.

Heart failure (HF) is a major and growing global public health problem, associated with substantial morbidity and death, including significant reductions in quality of life and a high symptom burden on a daily basis. ¹ , ² , ³ , ⁴ With this increasing burden on public health, there is a large unmet need for new HF device therapies to improve clinical care of these patients beyond what can be achieved by drugs. Traditionally, the development of medical devices for HF requires considerable time, effort, and resources, including pivotal randomized clinical trials to evaluate therapeutic benefit for regulatory review. The substantial time required by this process leaves many patients with HF limited treatment options and delayed access to devices that could potentially improve quality of life and reduce the likelihood of hospitalization and death.

In efforts to streamline the evaluation of medical devices for regulatory approval, the US Food and Drug Administration (FDA) issued a guidance document in 2015 that was commonly known as the Expedited Access Program. ⁵ This program was intended to provide patients and health care providers with timely access to medical devices by speeding up their development, assessment, and review, while preserving statutory standards for premarket approval. Following enactment of the 21st Century Cures Act in 2016, ⁶ the intent of the Expedited Access Program was further codified in an FDA guidance labeled the Breakthrough Devices Program, ⁷ which replaced the Expedited Access Program in 2018. The Breakthrough Devices Program allows for flexibility in trial design, end points, and statistical analysis, as well as a balancing of pre‐ and postmarket clinical evidence generation.

In 2019, a medical device known as the Optimizer Smart System (Impulse Dynamics, Marlton, NJ) was approved by the FDA under the Breakthrough Devices Program for the treatment of people with HF and moderately reduced left ventricular ejection fraction (LVEF). The device delivers cardiac contractility modulation (CCM) therapy, which is an electrical therapy for the heart that differs from cardiac pacing, as described in the subsequent section. The pivotal FDA regulatory trial for the Optimizer Smart System incorporated innovative Bayesian statistical methodology that leveraged external randomized trial results. This allowed the pivotal trial to be conducted with a smaller sample size than that achieved through traditional methods, accelerating the pathway to regulatory approval while maintaining the statutory standards for premarket approval.

The purpose of this manuscript is to explain the Bayesian statistical approach for evaluating CCM effectiveness in the FIX‐HF‐5C (Confirmatory Randomized Trial Evaluating the Optimizer System) clinical trial in the context of observed results, which provided a scientifically valid approach to leverage data from an external clinical trial to expedite therapeutic evaluation in a regulatory setting and may serve as a model for future clinical trials.

Clinical and Regulatory Background

Cardiac Contractility Modulation

CCM therapy is delivered from an implanted pulse generator via 2 right ventricular leads. The system delivers a high‐voltage (≈7.5 volts) biphasic electrical signal of long‐duration (≈20 milliseconds) during the absolute refractory period. ⁸ , ⁹ , ¹⁰ Because the electrical impulses are delivered during the refractory period, CCM signals do not result in a myocardial contraction but, over time, lead to structural and functional changes in the myocardium, including molecular changes, enhanced contractility, and decreased left ventricular volume. ¹⁰ Augmentation of intracellular calcium within the cardiomyocyte as a result of increased extracellular calcium influx and cycling by the sarcoplasmic reticulum likely explains the acute effects, while improvements in gene expression and posttranslational modification of proteins and pumps involved in calcium cycling contribute to longer‐term effects. ⁸

US Pivotal Trials of CCM

There were 2 pivotal randomized clinical trials evaluating the effectiveness of CCM plus optimal medical therapy versus optimal medical therapy alone:

1. FIX‐HF‐5 (original Randomized Trial Evaluating the Optimizer System): 428 patients with New York Heart Association (NYHA) class III and class IV HF and LVEF <45%. ¹¹

2. FIX‐HF‐5C: 160 patients with NYHA class III HF and baseline LVEF ≥25% and <45%. ¹²

The first trial, FIX‐HF‐5, failed to demonstrate superiority of CCM versus control on the prespecified primary analysis. However, a post hoc subgroup analysis showed evidence of CCM benefit in a clinically relevant subgroup comprising ≈50% of the total enrolled study population, defined as patients with NYHA class III HF and baseline LVEF between 25% and 45%, inclusive. These promising results led to a second pivotal study (FIX‐HF‐5C) to confirm CCM effectiveness within the selected subgroup. In collaboration with the FDA within the context of the Breakthrough Devices Program, the FIX‐HF‐5C primary analysis used Bayesian statistics to formally incorporate prior knowledge from the FIX‐HF‐5 subgroup analysis. Detailed results and conclusions of both trials are published elsewhere ¹¹ , ¹² ; a high‐level summary is provided below.

FIX‐HF‐5 Pivotal Trial

The pivotal FIX‐HF‐5 trial was an open‐label 1:1 randomized clinical trial comparing the effectiveness of CCM plus optimal medical therapy versus optimal medical therapy alone (control). Four hundred twenty‐eight subjects with NYHA class III (88%) and class IV (12%) HF with LVEF up to 45% were enrolled, with a primary end point of ventilatory anaerobic threshold. The primary effectiveness analysis compared the difference in ventilatory anaerobic threshold responder rates at 24 weeks for CCM (17.6%) versus control (11.7%) and failed to reach statistical significance (2‐sided P value=0.09). However, there were concerns by the steering committee about selection of the primary effectiveness end point, as ventilatory anaerobic threshold had never been used in a large multicenter randomized study before that trial. One specific weakness of the end point was the large amount of missing data (about 30%), which complicated the primary analysis and interpretation of results. Missing data was most frequently due to the exercise core laboratory reader's inability to determine a value for ventilatory anaerobic threshold, even when the patient performed an adequate cardiopulmonary stress test. ¹³

A key secondary analysis, difference in responder rates in peak oxygen consumption (VO₂; response defined as a 20% improvement from baseline), showed a 17.6% response rate in the CCM group and 13.6% in the control group (nominal 2‐sided P value=0.23). Additional secondary analyses demonstrated strong benefit in responder rates for both the Minnesota Living With Heart Failure Questionnaire quality‐of‐life score and NYHA class (nominal 2‐sided P value=0.003). Given these results, trial investigators conducted post hoc analyses investigating the benefit of CCM therapy on the mean change in peak VO₂. Including all subjects in the FIX‐HF‐5 trial, the 24‐week mean difference in peak VO₂ was 0.51 mL/kg per min greater in CCM therapy versus control (nominal 2‐sided P value =0.025). In a subset of the study population (N=229 or about half of the total study population), defined by LVEF ≥25% and NYHA class III, the 24‐week mean difference in peak VO₂ was 1.08 mL/kg per min greater in CCM therapy group versus control (nominal 2‐sided P value=0.002). This post hoc analysis generated the hypothesis for a new FIX‐HF‐5C confirmatory study ¹¹ in the identified subgroup population.

FIX‐HF‐5C Pivotal Trial

The FIX‐HF‐5C trial was subsequently designed to confirm the benefit in peak VO₂ of CCM therapy in patients with LVEF ≥25% and NYHA class III HF. In FIX‐HF‐5C, 160 patients were randomized 1:1 to either CCM plus optimal medical therapy or to optimal medical therapy alone (open‐label) with similar inclusion/exclusion criteria other than the LVEF cutoff as the FIX‐HF‐5 trial. Peak VO₂ is a well‐accepted endpoint in HF clinical trials, as it has intrinsic value to patients (improved exercise ability) and correlates with clinical outcomes.

Within the context of the Breakthrough Devices Program, both the sponsor and the FDA agreed on a FIX‐HF‐5C primary analysis plan that incorporated Bayesian borrowing of positive results from the corresponding FIX‐HF‐5 subgroup analysis. This enabled a smaller study than what traditionally would have been required (N=160 instead of N>230) by formally incorporating the prior data into the FIX‐HF‐5C primary effectiveness analysis.

Statistical Analysis

Because of the proprietary nature of the data used to conduct these analyses, the data are not available to the public. However, computer code for fitting the Bayesian model is available from the corresponding author upon request. In addition, both trials included in this analysis were approved by institutional review committees, and participating subjects gave informed consent.

FIX‐HF‐5C Primary Effectiveness Analysis

The primary effectiveness analysis of the FIX‐HF‐5C trial was a Bayesian repeated‐measures model, which incorporated baseline, 12‐week, and 24‐week peak VO₂ data in estimating the 24‐week mean difference in peak VO₂ between the CCM therapy and control groups. The primary analysis model borrowed results from the analogous model of the 229 patients from the FIX‐HF‐5 subgroup, but downweighted the FIX‐HF‐5 results by 70% (ie, a 30% weight, or ≈69 subjects worth of information) to ensure that the FIX‐HF‐5C conclusions were not overly influenced by the positive FIX‐HF‐5 data. ¹⁴ To fully understand the use of (downweighted) Bayesian borrowing in the FIX‐HF‐5C trial, it is important to have a general knowledge of key Bayesian concepts described in the subsequent section; additional tutorials are available in the literature. ¹⁵ , ¹⁶ , ¹⁷ , ¹⁸ In addition, a previous publication provided rationale and details on the original Bayesian methodology proposed for FIX‐HF‐5C ¹⁹ ; however, the statistical methods were subsequently modified with collaboration from the FDA and the sponsor as part of the Breakthrough Devices pathway (before unblinding any data), and this article is based on the revised analysis plan that ultimately was implemented on the FIX‐HF‐5C data.

Overview of Bayesian Statistical Methodology

Bayesian statistics are used to quantify the probability distribution of a parameter (eg, treatment effect) conditional on observed data. One first begins with a belief of likely values of the parameter before observing trial data, also known as a Bayesian prior distribution, which could be based on expert opinion or external/historical data (eg, previous FIX‐HF‐5 trial); see Figure 1A and 1B. These beliefs are positive and in total add up to 1, such that a belief for any particular region (eg, CCM is better than control) is calculated by adding up belief within that region. Next, the experiment is conducted, and new data are collected prospectively (eg, new FIX‐HF‐5C trial). The prior distribution is combined with the observed data to formulate a Bayesian posterior belief, which is known as the Bayesian posterior distribution. The Bayesian posterior distribution (Figure 1C) provides the basis of statistical inference (eg, estimation, hypothesis testing), where Bayes' Theorem ²⁰ is the mathematical connection between the posterior distribution, prior distribution, and observed data.

Figure 1. Observed data+Bayesian prior=Bayesian posterior.

A, Bayes' Theorem: mathematical connection between the posterior distribution, prior distribution, and observed data. B, Distribution of observed data; downweighted prior distribution. C, Posterior distribution, mean, 95% CI, and probability of superiority. FIX‐HF‐5 indicates the original Randomized Trial Evaluating the Optimizer System; and FIX‐HF‐5C, Confirmatory Randomized Trial Evaluating the Optimizer System.

The Bayesian posterior distribution provides an estimate of treatment benefit via the mean of the posterior distribution, with a corresponding credible interval to capture most (eg, 95%) of the posterior belief. In clinical trials, one key advantage of conducting statistical inference using a Bayesian posterior distribution is the ability to make probabilistic statements about a treatment superiority. For example, the Bayesian posterior probability of superiority in the FIX‐HF‐5C trial was 0.989, which exceeded the prespecified 0.975 threshold required to demonstrate superiority of device versus control.

In the context of evaluating the benefit of CCM therapy versus that of the control, a Bayesian prior distribution was used in the FIX‐HF‐5C primary analysis on the basis of external data from the previous FIX‐HF‐5 trial. The prior is combined with the FIX‐HF‐5C observed data to formulate a Bayesian posterior distribution for CCM therapy benefit in the FIX‐HF‐5C primary analysis. The FIX‐HF‐5C Bayesian posterior distribution provides an estimate of the treatment benefit, a corresponding 95% Bayesian credible interval, and the Bayesian posterior probability of treatment superiority versus control. For quick reference, a glossary of Bayesian terminology is provided in Table 1.

Table 1.

Glossary of Bayesian Terminology for Estimating a Treatment Effect

Bayesian term	Definition	Application
Prior distribution	The belief in a treatment effect before collecting data	Downweighted FIX‐HF‐5 data
Posterior distribution	The belief in a treatment effect after updating the prior belief with observed data	FIX‐HF‐5C primary analysis
Bayes' theorem	The mathematical connection between the posterior distribution, prior distribution, and observed data
Posterior mean	An estimate for the treatment effect based on the posterior distribution	FIX‐HF‐5C estimate: 0.836 mL/kg per min
Posterior probability	A metric for the probability of a parameter being within a specified range	Probability of CCM superiority, Prob(treatment effect>0)=0.989
Bayesian credible interval	Interval from posterior distribution that provides likely values of the treatment effect	FIX‐HF‐5C 95% BCI: 0.123–1.522
Exchangeability	Assumption regarding lack of systematic differences in treatment effects between 2 trials	FIX‐HF‐5C vs FIX‐HF‐5

BCI indicates Bayesian credible interval; and CCM, cardiac contractility modulation.

A full description of Bayesian methodology and comparison to traditional frequentist methods is outside the scope of this article; rather, the purpose of this high‐level overview is to enable a better understanding of the Bayesian borrowing of external clinical trial data ²¹ used in the FIX‐HF‐5C trial and provide a model for future pivotal trials in similar circumstances.

FIX‐HF‐5C Justification for Bayesian Borrowing

There are both clinical and statistical justifications for including Bayesian borrowing in the FIX‐HF‐5C primary analysis plan. The FDA Breakthrough Devices Program designation allowed for the exploration of more flexible and innovative designs to accelerate the development of novel technology intended to treat a life‐threatening condition. In addition, there was a biological rationale for the subgroup effect. In brief, CCM therapy had been shown to provide a modest increase of left ventricular contractility ²² and so might be expected to work better in patients with mild to moderate left ventricular dilation and systolic dysfunction compared with those with severe left ventricular dilation and dysfunction.

A key statistical assumption often underlying models with Bayesian borrowing is study exchangeability. ²³ Two parameters (eg, treatment effects) from 2 distinct studies are exchangeable if there is no prior reason to suspect systematic differences between the parameters between the trials in any given direction, and there is no ability to predict that the new study will have a larger or smaller parameter (eg, treatment effect) than that of the previous trial. When present, this exchangeability assumption is captured in the prior distribution, and there is no requirement that the posterior distribution has this property. Although having a treatment effect that satisfies the exchangeability assumption would be sufficient for statistical justification of the FIX‐HF‐5C primary analysis plan, the selected Bayesian methodology for FIX‐HF‐5C relied on a less stringent assumption: the prior belief (based on FIX‐HF‐5) accurately reflects available information about the intended population. This is largely a clinical assumption, with justification arising from very similar trial protocols investigating the same device (albeit different sample sizes), with the same inclusion/exclusion criteria for the subgroup population, overlapping sites, identical blinded core labs, identical adjudication committees, and small temporal gap between studies, such that both studies are believed to be estimating the same parameter (24‐week mean change in peak VO₂) in the identified subgroup population. These considerations led to clinical and statistical agreement between the FDA and the sponsor in the Bayesian prior distribution.

FIX‐HF‐5C Prior Distribution

In the FIX‐HF‐5 study, the Bayesian model‐based mean estimate of CCM therapy peak VO₂ benefit in the identified subgroup was 1.08 mL/kg per min (standard error=0.344), with a 95% Bayesian credible interval of 0.413 to 1.759. The Bayesian posterior probability of CCM superiority versus control was 0.999. However, because this was not the prespecified primary analysis of FIX‐HF‐5, there was reasonable doubt regarding the magnitude of CCM benefit in this subgroup population. Hence, these results were downweighted by 70% to be used as a prior distribution in the subsequent FIX‐HF‐5C trial. Figure 2 shows the downweighted distribution relative to the observed FIX‐HF‐5 posterior distribution. Conceptually, the FIX‐HF‐5 posterior distribution is flattened to create a wider distribution with greater uncertainty, resulting in a mean of 1.08 and a larger standard error of 0.628. This downweighted distribution uses data from all 229 subjects, but reduces the relative contribution or weight of each individual. This results in an approximate effective sample size of 69 patients, or 30% weight of the 229 subgroup sample size as the Bayesian prior distribution in the confirmatory FIX‐HF‐5C trial.

Figure 2. Bayesian downweighted FIX‐HF‐5 posterior.

FIX‐5/FIX‐HF‐5 indicates the original Randomized Trial Evaluating the Optimizer System.

The incorporation of positive CCM data from the FIX‐HF‐5 trial in the FIX‐HF‐5C primary analysis both (1) increases statistical power; and (2) increases the type I error rate of the FIX‐HF‐5C analysis. In the FIX‐HF‐5C trial, a downweighting of 70% was chosen to control the one‐sided type I error rate under 0.10, which was deemed by the FDA to be an acceptable risk for this disease and patient population in the context of the Breakthrough Devices Program. In general, the magnitude of downweighting and acceptability of type I error rate inflation is typically a case‐by‐case determination, and will depend on the disease, trial design and methodology, strength of external data, confidence in the selected prior distribution, and safety profile of the treatment being evaluated. These factors all have a direct impact on the magnitude and justification of any sample size reduction, which may not always be appropriate. Additional details of the downweighting of external data for FIX‐HF‐5C are provided in Data S1.

Results

The FIX‐HF‐5C Bayesian primary analysis with FIX‐HF‐5 borrowing (70% downweighted) showed a mean improvement of 0.836 mL/kg per min with a 95% Bayesian credible interval of 0.123 to 1.522. The Bayesian posterior probability of superiority (with borrowing) equaled 0.989, which exceeded the prespecified threshold of 0.975 to reject the null hypothesis and conclude CCM superiority versus control. The model‐based estimated means in peak VO₂ across time via the Bayesian repeated‐measures model are provided in Figure 3. The model‐based estimated mean treatment difference at 24 weeks is shown in Figure 4 and Table 2.

Figure 3. Bayesian model‐based means and 95% CIs for peak VO₂.

*Points denote means; bars at each time denote 95% Bayesian credible intervals. VO₂ indicates oxygen consumption.

Figure 4. Bayesian model‐based treatment difference peak VO₂ at 24 weeks (mL/kg per min).

FIX‐HF‐5 indicates the original Randomized Trial Evaluating the Optimizer System; FIX‐HF‐5C, Confirmatory Randomized Trial Evaluating the Optimizer System; and VO₂, oxygen consumption.

Table 2.

Bayesian Model Estimated 24‐Week Mean Difference in Peak VO₂ by Study

Analysis	Treatment difference (mL/kg per min)	Standard error	Lower 95% BCI	Upper 95% BCI	Bayesian posterior probability
FIX‐HF‐5 down‐weighted	1.080	0.628	−0.151	2.311	0.957
FIX‐HF‐5C alone	0.799	0.459	−0.101	1.697	0.960
FIX‐HF‐5C with borrowing	0.836	0.364	0.123	1.552	0.989

BCI indicates Bayesian credible interval; FIX‐HF‐5, original Randomized Trial Evaluating the Optimizer System; FIX‐HF‐5C, Confirmatory Randomized Trial Evaluating the Optimizer System; and VO₂, oxygen consumption.

For more straightforward comparisons, Bayesian analyses were conducted on the FIXH‐HF‐5C data using a large prior variance to provide a Bayesian stand‐alone estimate without borrowing. The stand‐alone Bayesian analysis of FIX‐HF‐5C with 160 patients (Table 2) showed a mean improvement of 0.799 (95% Bayesian credible interval, −0.101 to 1.697) mL/kg per min for CCM versus control arm, with a Bayesian posterior probability of CCM superiority of 0.960, which failed to meet the superiority criteria of 0.975. The stand‐alone Bayesian approach closely aligned with a traditional frequentist estimate of the treatment effect, which equaled 0.793 (1‐sided P=0.041) and failed to meet the traditional 0.025 one‐sided significance level. Hence, the Bayesian borrowing was critical to the conclusion of CCM superiority.

With respect to estimation, the FIX‐HF‐5C Bayesian posterior mean with borrowing was a compromise between the observed FIX‐HF‐5C data mean and the downweighted prior mean from FIX‐HF‐5 (see Table 2 and Figure 5). In addition, the precision of the FIX‐5C posterior distribution with borrowing was greater than that of either of the FIX‐HF‐5C stand‐alone analysis or the downweighted prior belief separately, showing less uncertainty regarding the mean CCM benefit when combining the observed data with the prior distribution.

Figure 5. FIX‐HF‐5C Bayesian posterior with borrowing.

FIX‐HF‐5C indicates the Confirmatory Randomized Trial Evaluating the Optimizer System.

The effective sample size of the FIX‐HF‐5C primary analysis was ≈230 subjects, reflecting an increase of ≈70 patients worth of information relative to the stand‐alone N=160 analysis. Sensitivity analyses demonstrated that the prespecified Bayesian posterior probability superiority threshold of 0.975 would be satisfied with downweighting of FIX‐HF‐5 of 89% or less. Additional sensitivity analyses were conducted to evaluate the impact of deviations from other model assumptions, including various strategies for handling missing data, which did not change the conclusions of the primary analysis. ²⁴

The Optimizer Smart System was presented to the FDA Circulatory System Devices Panel in December 2018. Following extensive discussions, the panel voted 11 “yes” and 2 “no” on whether there is reasonable assurance of effectiveness in the study population. ²⁴ The overall vote on approval based on a favorable benefit‐to‐risk ratio was 12 in favor with 1 abstention. FDA approval was granted in March 2019.

Discussion

Given that the targeted subgroup (patients with LVEF between 25% and 45% and class III NYHA status) was selected from a post hoc analysis of FIX‐HF‐5, there is a risk of bias in the FIX‐HF‐5 analysis that may be propagated to the FIX‐HF‐5C primary analysis. In other words, one could argue that the FIX‐HF‐5 subgroup result was cherry picked among many possible subgroups, and the increased benefit observed in that specific subgroup is due to chance alone. This is a valid statistical concern that can lead to biased estimates in a subgroup population. In fact, most failed clinical trials produce subgroups with seemingly promising results; such results are expected by chance alone even when there is no underlying treatment benefit in the patient population. The degree of belief in subgroup findings depends on the statistical methods (eg, prespecified versus post hoc analyses), the number of subgroups explored, and the clinical rationale for the subgroup effect, that is, the biological plausibility of the identified subgroup having greater benefit. In the context of FIX‐HF‐5C, a Bayesian analysis with 30% borrowing on the full FIX‐HF‐5 data set (N=428) would also demonstrate superiority with the prespecified criteria (results not shown); such an analysis does not suffer from an increased risk of bias from post hoc subgroup selection. Regardless, a prospective plan to borrow from an external post hoc subgroup analysis needs careful consideration and scrutiny due to the increased probability of false‐positive results and potential bias arising from subgroup analyses. In addition, the inclusion of known/observed favorable data can raise doubts regarding the assumption of exchangeability across studies required by some Bayesian methods. We recommend careful adherence to the statistical principles outlined in the 2010 FDA guidance for the use of Bayesian statistics in medical device trials ²³ and prospective discussions with the FDA to address both clinical and statistical justification for such methods.

Although the sponsor and the FDA reached agreement on the FIX‐HF‐5C prior distribution, that does not necessarily imply that all people (eg, experts, physicians, patients) agree with the chosen prior distribution. This was evident during the FDA advisory committee panel, where additional scrutiny was given to the FIX‐HF‐5C trial conclusion because of the reliance on the FIX‐HF‐5 data. In addition, there was considerable panel discussion on whether a peak VO₂ estimated mean difference of 0.836 mL/kg per min between CCM and control is clinically meaningful. Although viewed by many as a modest benefit, the FDA panel of experts ultimately voted it was meaningful to patients with a life‐threatening condition that lacked an FDA‐approved alternative therapy. ²⁵ Moreover, the improvement in peak VO₂ observed in the FIX‐HF‐5C trial of HF with narrow QRS interval was comparable to that observed with cardiac resynchronization therapy in HF with wide QRS interval, which is a well‐established treatment for HF. ²⁶

As an alternative to Bayesian borrowing, FIX‐HF‐5C trial investigators could have explored the idea of using traditional (frequentist) strategies with P values for testing the FIX‐HF‐5C stand‐alone treatment effect at a higher nominal α level (eg, 1‐sided 0.10). Such approaches lack the ability to formally incorporate external data into the estimated treatment effect and corresponding hypothesis test. In contrast, the Bayesian methodology (1) formally incorporates the external data into the hypothesis testing framework; (2) allows quantification of the magnitude of borrowing with respect to sample size; and (3) formally incorporates the external data into the estimated treatment effect and corresponding Bayesian credible interval (eg, for labeling purposes), which has greater precision than a traditional stand‐alone analysis.

More generally, Bayesian borrowing is not limited to the specific context of borrowing from an estimated treatment effect on the basis of a subgroup in a previous trial. Bayesian borrowing methodology can be used to leverage control arm data from other randomized clinical trials, leverage the effect of a therapy/device across different subtypes of a disease, or leverage natural history data to evaluate a therapy/device relative to the expected disease progression. ²¹ Although the specific statistical methodology for Bayesian borrowing will depend on the specific context of a given trial, the same general Bayesian concept can be applied, that is, leverage external data via a Bayesian prior, collect new data, and formulate a Bayesian posterior to evaluate a therapeutic intervention. In addition, trial designs incorporating Bayesian borrowing could also explore alternatives to formal hypothesis testing, for example, Bayes factors that compare the posterior odds of treatment superiority versus the null hypothesis. ²⁷

The FIX‐HF‐5C clinical trial design effectively leveraged results from a prior external randomized clinical trial (FIX‐HF‐5), providing a model for other devices in similar circumstances. This was done through Bayesian statistical methodology, which provided a scientifically valid approach to prospectively incorporate the prior subgroup data into the FIX‐HF‐5C pivotal trial's primary analysis plan. The resulting Bayesian treatment effect appropriately leveraged available data from both trials to provide a more precise estimate of therapeutic benefit in this patient population (Figure 1). A key feature of the FIX‐HF‐5C primary analysis was that the analysis and Bayesian borrowing methodology were completely prespecified. Regulatory involvement was critical in the design stage to ensure scientific credibility and compliance with the FDA guidance on Bayesian Statistics in Medical Device Clinical Trials. ²³ Under the FDA Breakthrough Devices Program, the borrowing of prior external data resulted in a smaller pivotal trial than otherwise would have been conducted, ultimately leading to faster FDA approval and earlier patient access to a device that improves the quality of life of patients with no other approved therapy.

Sources of Funding

None.

Disclosures

Dr Saville reports consulting fees paid to his employer (Berry Consultants) by Impulse Dynamics for statistical advising. Dr Burkhoff reports consulting arrangements with Impulse Dynamics and Corvia Medical. Dr Abraham has received speaker honoraria from Impulse Dynamics.

Supporting information

Data S1–S2

Table S1

Reference 28