Overcoming Regulatory Aversion to Novel Methods of Evidence Generation

Drug development remains high‐risk, long, and expensive. Regulators share industry's interest in facilitating access to new medicines that will improve public health. This commentary accompanies articles by leaders from the European regulatory community that make the case for prospective validation of novel methods, such as the use of synthetic nonconcurrent control arms, Bayesian borrowing to reduce sample sizes, and the use of real‐world data (RWD) to support regulatory decision making.

Controlled clinical trials are the accepted gold standard for generations of evidence to inform regulatory decision making. The characteristics of the controlled trial, such randomization, blinding, prospective choice of end points, and statistical analytical plans, and other bias control measures are essential features that we take for granted as defining that gold standard. Has it always been that way? In fact, the modern randomized controlled trial (RCT) evolved after time and was catalyzed by regulatory requirements requiring preapproval demonstration of drug effectiveness as well as safety. Subsequent to the passage of the 1962 amendments to the Food Drug & Cosmetic Act (FD&C), much effort was devoted to understanding the evidentiary requirements for meeting that standard. The Drug Efficacy and Safety Initiative (DESI) was a major initiative that examined the available evidence supporting the safety and effectiveness of drugs that had reached the US market prior to the 1962 requirements and greatly informed the US Food and Drug Administration (FDA) policy on the effectiveness standards culminating in regulations that defined the features of adequate and well‐controlled trial(s) required as evidence in support of marketing applications.1

The challenge, however, is to write and enact regulation that permits sufficient regulatory flexibility to allow for durability and relevance of the rules, to encourage creativity, and to not be tethered to precedents that impede innovation. Modernization of regulations and the evolution of regulatory standards are required to effectively support innovation and, not infrequently, are driven by public and political expectations, as has been seen with significant amendments to the Federal FD&C Act in 1997 (FDA Modernization Act (FDAMA)), 2007 (FDA Amendments Act (FDAAA)), and the 2012 (FDA Safety and Innovation Act (FDASIA)).1

Importantly, it should be remembered that drug development is now largely a global effort and, for a pharmaceutical industry striving to develop and gain global market access for their products, is a challenge that could be exponentially compounded by disharmonized regional and national regulatory requirements. Most recently, regulators have been hearing calls to partner with patients, providers, and industry in innovating ways to speed access to novel therapeutics that address unmet medical needs. Facilitated regulatory pathways, such as accelerated approval, priority review, and fast track, and breakthrough designation in the United States, conditional approval and PRIME in Europe, and Sakegake designation in Japan are but a few examples of speeding development, review, and approval timelines.2 More recently, the concept of adaptive biomedical innovation and adaptative licensing has emerged and is now being piloted by the European Medicines Agency (EMA), and there is an increased focus on novel methods for generating evidence or use of alternative sources of empirical evidence, such as RWD to support regulatory decision making.2, 3

This issue of Clinical Pharmacology & Therapeutics (CPT) contains an important paper by prominent leaders from the European regulatory community and articulates some familiar concerns regarding alternative sources of evidence to support regulatory decision making.4 Eichler et al.⁴ articulate some of the recent and rapid availability of healthcare data that present huge opportunities to address medical research questions or mind for healthcare insights; however, they also identify well‐known limitations and obstacles such as:

• Operational readiness factors, such as breadth of population coverage, interoperability of health data systems, and data quality and completeness

• Data governance readiness factors, such as privacy concerns, level of required consent, and other legal and intellectual property issues

The authors are optimistic that these aforementioned issues can be overcome over time but remain concerned that, if they are, the evidence sources and analytical methodologies remain unvalidated and “the uncritical adoption of novel methodologies may lead to false conclusions, poor healthcare decisions, and ultimately patient harm.” This is a familiar refrain, akin to the precautionary principle, framed as “methodology aversion in drug regulation,” and refreshingly acknowledged by the authors as to having a potentially chilling effect on regulatory innovation.

In our opinion, the real value of the Eichler et al.⁴ paper is that it identifies multiple examples of novel methodologies, their current limitations, and specific proposals on how to validate them for future use in informing regulatory decision making. The opportunities range from borrowing of data from prior experience to reduce sample sizes in subsequent confirmatory RCTs, to the bold idea of replacing RCT data by RWD analyses to modeled extrapolation of RCT data to unstudied populations and others. None of the individual proposals are necessarily novel to the readers of CPT, but the value of the paper is providing a case for the importance of validation, a proposed framework for working with industry and other stakeholders to achieve “fit for purpose” validation, and highlighting the EMA's collaborative platforms that are available for such activities.

In our experience, the EMA and global regulators have been open to novelty in clinical trial design, statistical approaches, and modeling/extrapolation methods. However, there are some practical challenges to the general proposals offered by Eichler et al.⁴ Although observational methods have improved greatly, the biases that are intrinsic and residual to observational studies can never be completely eliminated and, as such, can really not be expected to replace RCTs as a gold standard. Better practices, such as prospective specification of study and analytical methods and end points, replication of results in complementary or expanded populations and across data sources, and generally accepted ways of dealing with missing data and/or data quality issues will be important and necessary. However, the real question is whether they can, when rigorously executed, provide adequate and sufficient evidence to support regulatory decision making. The answer to that question is, at this time is…..it depends. From a regulator's perspective, it always depends on the residual uncertainty that is left when any evidence is provided for a preapproval decision. If the disease is deadly and no alternative therapies exists, health authorities have already demonstrated willingness to accept small datasets and end points that are surrogates or reasonably likely to predict clinical benefit for the patient. On the other hand, for therapeutic areas with a broad array of existing safe and effective therapies, regulators often expect larger preapproval safety datasets with a more precise understanding of the benefits and comparative benefits of the new therapy. It seems that regulators will also consider nonrandomized evidence, extrapolation to unstudied populations, and indirect benefit‐risk comparisons in the same context.

Ultimately, the acceptance of novel methods and data sources will be the result of collaboration of academia, industry, and health authorities. There are several noteworthy examples, including the accelerated approval of blinatumomab in the United States and Europe for treatment of relapsed/refractory Philadelphia chromosome‐negative acute lymphoblastic leukemia based on a single‐arm, open‐label, study based on a comparison to patient level data from chart reviews of historical control patients receiving standard of care. The full approval for this indication was ultimately granted on confirmatory phase III study data.5 This example illustrates the importance of medical context (unmet medical need) as well as the value of corroboration across sources of evidence in validating methods. A broader effort is being led by Franklin et al. in prospectively evaluating nonrandomized observational RWD analyses against an RCT result is ongoing.6

As we have seen from experience with RCTs, the current “gold standard” evolved over time and informed by experiential learning and collaboration of stakeholders. The RCT continues to evolve and examples of such include refined concepts of clinical trial adaptation, insights into methods to better handle postrandomization intercurrent events through better articulation of design estimands, the use of Bayesian statistics to borrow priors from previous clinical experience resulting in optimization of subsequent trial sample sizes, and the increased use of nonconcurrent “synthetic” control arms. Transparent dissemination of the outcomes of case studies, such as blinatumomab as well as Franklin's systemic assessment, will be useful in informing regulatory practice, delineating the expectations for “fit for purpose” validation, and incentivizing observational and other novel approaches in generating evidence for regulatory decision making. As Eicher and colleagues state, “the stakes are high—overcoming methodology aversion and ensuring that all stakeholders arrive at a nuanced view between categorical rejection and naïve adoption of such methods.”

Funding

No funding was received for this work.

Conflict of Interest

Drs. Baumfeld Andre and Honig are employees of Pfizer and any opinions expressed here are their own and do not represent the views of Pfizer, Inc. As an Associate Editor for Clinical Pharmacology & Therapeutics, Peter K Honig was not involved in the review or decision process for this paper.