Real World-Evidence for Regulatory Use Decision Aid: An Interactive Tool To Inform Clinical Development and Regulatory Strategies

Abstract

Real-world evidence (RWE) is increasingly used to complement clinical trial data for regulatory decision-making and in certain cases utilized to establish the clinical effectiveness of a therapy. However, the use of RWE is not applicable for all regulatory submissions, and it can be challenging to identify appropriate use cases. An interactive tool was developed (“Decision Aid,” https://sn.pub/TpDjZx) to assist researchers, industry, and other stakeholders in identifying regulatory situations that can benefit from leveraging RWE by organizing precedent cases based on a given regulatory objective (new product approval, labeling expansion for new indication or additional clinical data, post-marketing requirement) and type of RWE study design (external control, observational study, pragmatic trial). Key success factors ensuring fit-for-purpose data and rigorous methods (e.g., clear endpoints, minimizing bias, data completeness) are also described. The tool allows the user to navigate through the precedent cases by selecting certain regulatory objectives and/or study designs. The Decision Aid supports regulatory activities in the RWE space and encourages further use of RWE in regulatory decision-making.

Supplementary Information

The online version contains supplementary material available at 10.1007/s12325-022-02257-4.

Key Summary Points

The aim of this paper is to describe an interactive Decision Aid designed to:

Serve as a tool for researchers, industry, and other stakeholders to identify areas of opportunity for real-world evidence (RWE) to support regulatory strategies and clinical development plans.

Illustrate key uses of RWE for regulatory purposes (by utilizing precedent cases where RWE was successfully used to support regulatory decisions), corresponding RWE study designs, and underlying key success factors.

Encourage further use of RWE and activity in this space to add to the growing evidence and establishment of RWE in regulatory submissions.

This manuscript and associated Decision Aid are intended to help researchers identify potential use cases in which RWE might be considered to inform regulatory decision-making.

Interactive Decision Aid

This article contains an interactive Decision Aid  to help researchers, industry, and other stakeholders in identifying regulatory situations that can benefit from leveraging real-world evidence. To view it, click here: https://sn.pub/TpDjZx. The interactive Decision Aid can also be accessed through the supplementary material. All files must be extracted for full functionality.

Introduction

Real-world evidence (RWE) has traditionally played a relatively limited role in the regulatory process, for example, addressing unmet need and satisfying post-marketing safety monitoring requirements. In the current evolving healthcare landscape, however, there has been a shift towards utilizing RWE in a more central and fit-for-purpose role through identifying situations in which RWE can help establish the clinical effectiveness of a therapy to support regulatory decision-making rather than only to supplement insights from clinical trials [1].

In the US, the Framework for the Food and Drug Administration’s (FDA’s) RWE Program was released in December 2018 and outlines several important RWE-related efforts the Agency is undertaking to evaluate the potential of RWE to support changes to labeling, such as adding or modifying an indication, a change in dose, dose regimen, or route of administration; adding a new population; or adding comparative effectiveness or safety information [2]. In Europe, the European Medicines Agency’s (EMA’s) Regulatory Science Strategy 2025 initiative includes the promotion of high-quality RWE in decision-making to generate complementary evidence across the product lifecycle as one of its five strategic goals [2]. RWE initiatives are being conducted by other global health authorities around the world as well, including in Canada [3], China [4], and Japan [5], to encourage and optimize the use of RWE for regulatory decisions, while focusing on the reliability and quality of evidence. Given the increasing interest in leveraging RWE for regulatory decision-making, there have been many recent publications on insights and recommendations on ways to ensure fit-for-purpose RWE [6–9]. In addition, the currently ongoing COVID-19 pandemic has further reinforced and accelerated Health Authorities’ interest in the potential utility of RWE [10–12].

Despite the growing interest in leveraging RWE by industry and evaluation of RWE by Health Authorities for regulatory decision-making, it is important to note that RWE is not applicable to all situations and should not be considered a replacement for clinical trials [13]. Regulatory acceptance of RWE to support data from clinical trials is subject to the respective Health Authority’s approaches and practices. The authors of this manuscript developed an interactive tool (“Decision Aid”) to assist in more easily identifying circumstances in which the use of RWE informed regulatory decision-making (Figure 1; https://sn.pub/TpDjZx). To date, RWE has been used in certain situations to support new product approvals, label expansions for new indications or additional clinical data, and fulfillment of post-marketing requirements. In such cases, various real-world (RW) or RW-clinical hybrid designs have been utilized, including external controls (external benchmarks or external comparators), observational studies, and pragmatic clinical trials. There are various factors, such as the rarity of the disease, magnitude of treatment effect, availability of quality RWD, and others, that can help inform the appropriateness of a RWE approach for regulatory decision-making. These factors can be better understood by reviewing precedent cases in this space.

The aim of this paper is to describe an interactive Decision Aid designed to:

• Serve as a tool for researchers, industry, and other stakeholders to identify areas of opportunity for RWE to support regulatory strategies and clinical development plans.

• Illustrate key uses of RWE for regulatory purposes (by utilizing precedent cases where RWE was successfully used to support regulatory decisions), corresponding RWE study designs, and underlying key success factors.

• Encourage further use of RWE and activity in this space to add to the growing evidence and establishment of RWE in regulatory submissions.

This article and associated Decision Aid are intended to help researchers identify potential use cases in which RWE might be considered to inform regulatory decision-making.

Compliance with Ethics Guidelines

This article is based on previously conducted studies and does not contain any new studies with human participants or animals.

Methods

Several case examples in which RWE was leveraged to support successful FDA and EMA regulatory submissions were selected and described based on a systematic review of regulatory labels, published literature, and regulatory assessment reports. Each case study was matched with the regulatory objective obtained (new product approval, labeling expansion for new indication or additional clinical data, post-marketing requirement) as well as the type of RWE study design used (external control, observational study, pragmatic trial). Based on this matching, the Decision Aid was organized accordingly by regulatory objective and RWE study design type.

Case studies were assessed for key success factors that have been described in the context of ensuring fit-for-purpose data and rigorous methods, which contributed to a positive regulatory opinion. The assessment was based on a checklist to ensure regulatory-grade data quality (completeness, transparency, generalizability, timeliness, scalability) [14], supplemented by additional considerations around clear endpoint definitions and strategies for minimizing bias for a careful assessment of data quality, reliability, and relevance [2, 15, 16]. Providing clear definitions of primary and secondary endpoints is important to avoid variable documentation of key events potentially influenced by subjectivity. RWD often encompasses a broader spectrum of patients which leads to better generalizability than clinical trials, but potential biases need to be considered and mitigated to allow for adequate interpretation of results. Pre-defined data abstraction, harmonization, and quality monitoring are key markers of quality, integrity, and completeness. Being transparent with Health Authorities around the study design deployed, whether in protocols, analysis plans, or reporting of study results, is another key success factor to allow Health Authorities to rigorously assess the RWE submitted. Leveraging recent and timely RWD ensures the data provide relevant insights around clinical decision-making and adequately reflects the dynamic nature of routine medical practice and treatment use. For data that were intended for scaling, data curation processes, applicability to multiple contexts, and variable definitions were also assessed.

The Decision Aid includes case studies on pharmaceutical products from multiple manufacturers and includes case studies that were available at the time this manuscript was drafted. The authors intend to submit annual updates as the regulatory landscape, frameworks, and case studies change.

Regulatory Use Decision Aid

The Decision Aid can be accessed at https://sn.pub/TpDjZx.

Users can navigate through the Decision Aid by using sequential multi-step navigation as the tool matches the regulatory objective being evaluated with the corresponding key success factors, the most applicable RWE study types, and real-life precedent cases for reference.

• When opening the tool, the general RWE success factors are described, key terms are defined, and an overview of the case studies is provided.

• When clicking on a particular regulatory objective (new product approval, label expansion, and post-marketing requirement), the most applicable corresponding RWE study types are displayed, including external control, observational study, or pragmatic clinical trial.

• After selecting a specific RWE study type, the tool will display corresponding real-life precedent case studies for that specific combination of regulatory objective and study type, covering multiple therapeutic areas.

  • o
    Individual case examples include key information, such as narrative and relevant success factors.
  • o
    Each case study also includes hyperlinks to regulatory assessment reports to facilitate further reading.

• The user can navigate forward and backward throughout the tool to access all information.

The list of case studies included in the Decision Aid is not exhaustive, as new case studies potentially became available after the tool was developed.

Discussion

The Decision Aid will help guide pharmaceutical companies and clinical researchers in assessing general appropriateness of a potential RWE approach for use in clinical development within a regulatory context. The Decision Aid does not aim to replace Health Authority guidance or the critical importance of engaging Health Authorities on RWE proposals early on to gather their feedback on the proposed approach, but the tool can help identify potential opportunities for leveraging RWE to inform certain regulatory objectives. It is this identification step that the Decision Aid aims to focus on. Further comprehensive assessment based on the particulars of a situation may be necessary to confirm whether a real-world evidence-based approach for a particular scenario is appropriate.

Conclusion

RWE has been increasingly used in later stages of drug development. Importantly, the intent of RWE is not to replace clinical trials but to apply RWE where appropriate to support drug development and ultimately regulatory submissions. Through identifying and illustrating key aspects of situations where RWE has been utilized for regulatory decision-making, the Decision Aid helps pharmaceutical companies and clinical researchers understand the type of potential evidence necessary for regulatory submissions, adds to the knowledge of available RWE, and encourages further regulatory activity using RWE to pave the way for broader acceptance by clinical researchers, pharmaceutical companies, and Health Authorities. To maximize the chances for success, any research proposal that intends to be used for regulatory decision-making should be discussed with the relevant Health Authority before the research is started.

Supplementary Information

Below is the link to the electronic supplementary material.

Appendix 1: Glossary

External Control

External control data is any data generated from patients external to the parameters of the parent trial and used to provide context to a single-arm study where it would be impractical or ethical to design the study with a placebo or active comparator arm [17].

External Benchmark

A subtype of external control, in which real-word populations are used to provide context on outcomes. In this case, no direct comparison is made with the trial data, the data is aggregated.

External Comparator

A subtype of external control, in which patient-level RWD are used for a comparison versus the clinical data.

Observational Study

A type of study in which individuals are observed or certain outcomes are measured. No attempt is made to affect the outcome (for example, no treatment is given) [18].

Pragmatic Trial

Pragmatic trial is designed to test interventions in the full spectrum of everyday clinical settings in order to maximize applicability and generalizability [19].

Bias

In a scientific research study or clinical trial, a flaw in the study design or the method of collecting or interpreting information. Biases can lead to incorrect conclusions about what the study or clinical trial showed [18].

Real World Evidence (RWE)

RWE is the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD) [1].

Real World Data (RWD)

RWD are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources [1].

Effectiveness (in Contrast to Efficacy)

While RCTs provide evidence of therapeutic efficacy in a controlled setting, real-world studies produce evidence of therapeutic effectiveness in real-world practice settings [20].

RW-Clinical Hybrid Design

RW-clinical hybrid design is an integration of a traditional clinical trial with pragmatic design aspects to collect RWD on patients (21).