Real World Data and Evidence: Support for Drug Approval

Introduction

In 1962, Congress passed the Kefauver–Harris Amendments to the Food, Drug, and Cosmetic Act, which required manufacturers to provide evidence of effectiveness from adequate and well controlled studies before drug approval. Since then, the randomized, controlled trial (RCT), usually using data collection methods outside of conventional health care delivery systems, has been widely used to establish drug safety and effectiveness and support drug approval. Although RCTs may be considered a gold standard for evidence generation, concerns have been raised about the costs of conducting such trials and the generalizability of their findings to patients. These limitations of RCTs coupled with advances in science and technology have piqued interest in the use of real world data to generate evidence needed to support drug approval.

Real world data are defined as data relating to patient health status or the delivery of health care routinely collected from a variety of sources, including electronic health records, administrative and medical claims databases, or patient registries (1). Real world data have long been used to facilitate the conduct of clinical trials and understand the safety of a drug product after its introduction to the market. To date, however, real world data have not been widely used to support regulatory decisions about the effectiveness of a drug product.

In December 2016, Congress passed legislation requiring the Food and Drug Administration (FDA) to establish a program to evaluate the potential use of real world evidence to support the approval of a new indication for an approved drug and address postapproval study requirements. The FDA’s Real-World Evidence Program, established to address this requirement, “will evaluate the potential use of real world evidence to support changes to labeling about drug product effectiveness, including adding or modifying an indication, such as a change in dose, dose regimen, or route of administration; adding a new population; or adding comparative effectiveness or safety information” (1). In this perspective piece, we discuss the FDA’s progress in creating a framework for its real world evidence program and important concepts that have emerged in discussions to date. We also consider potential applications to the development of therapies for patients with kidney diseases.

A Framework for the FDA’s Real World Evidence Program

In the framework for its real world evidence program, the FDA draws a distinction between real world data and real world evidence. As noted above, real world data are defined as data relating to patient health status or the delivery of health care routinely collected from a variety of sources. Real world evidence is defined as clinical evidence about the usage and potential benefits or risks of a medical product derived from analyses of real world data (1). The framework makes this distinction because evaluation of real world evidence will need to take into consideration both the methodologies used to generate evidence and the reliability and relevance of the real world data used in analyses. The FDA’s framework, released in December 2018, addresses both considerations related to assessing real world data as well as study designs using real world data to support effectiveness.

Study Designs Using Real World Data

Perhaps the most promising future application of real world data to support effectiveness determinations is the integration of randomized clinical trials into conventional health care systems. Such trials could use data collected from electronic health records, laboratory or claims data, or emerging technology (actigraphy and mobile applications) to assess study outcomes or otherwise streamline the conduct of the trial (e.g., by facilitating recruitment and enrollment). Although much work remains to ensure that the data are sufficiently reliable, complete, and high quality, this approach holds promise for improving patient recruitment and retention, decreasing burden on study participants, increasing study efficiency, and possibly, better ensuring that trial populations are more representative of the populations likely to use a drug in clinical practice. The FDA is developing guidance on data quality issues unique to the real world data setting and related study design considerations.

Although the FDA has experience using observational (noninterventional) study designs in the postmarket setting to evaluate product risks in broad populations, using nonrandomized methodologies to determine effectiveness can be problematic because of concerns about the ability to adequately control for confounding, selection bias, and the possibility of misclassification error. Other important considerations when designing studies using real world data include appropriate comparator selection and how to handle patients who “crossover” to other marketed therapies or discontinue the study drug altogether. The FDA is currently exploring when the use of observational data may be appropriate for evaluating effectiveness from a broad policy perspective and also, as individual drug manufacturers submit proposals for using real world data to support marketing applications. Data standards will need to be developed to maximize integration of real world data into the evidence base used to evaluate marketing applications. Furthermore, the FDA is participating in a project (https://www.ctduplicate.org/) that will attempt to replicate results of large RCTs using noninterventional methodologies and electronic claims data to better understand which study designs and analytic approaches increase the likelihood of obtaining a valid result from such studies.

Key Concepts

Several key concepts have emerged in discussions with the larger community about the use of real world evidence to support regulatory decisions about the effectiveness of a drug product. One is the importance of ensuring sufficient data quality to enable reliable decisions, recognizing that it may be difficult for real world data to meet the same standards as data collected in traditional clinical trials. Another is the need to gain a better understanding of the study designs and analytic approaches using real world evidence that are most likely to provide the level of certainty and scientific rigor needed to address regulatory questions. Potential barriers to using electronic health records and insurance claims data, widely available and used sources of real world data, also exist. These include lack of system interoperability and data standardization, missing information on important confounders (smoking, body mass index, and degree of kidney impairment), and concerns about privacy. There is also widespread recognition that there will be a learning curve and that stakeholder engagement and demonstration projects will be critical.

Applications to Kidney Disease

There is unmet need for treatments that meaningfully improve the lives of patients with kidney disease, and real world data and real world evidence provide a possible means to accelerate the development of such therapies. Many have noted the potential to integrate RCTs into the data-rich clinical care setting of patients on dialysis, and efforts are underway to understand and address current challenges to doing so (2). In RCTs of patients with common kidney diseases, such as diabetic kidney disease, real world data could be used to facilitate the identification and enrollment of patients into trials. Study methodologies aimed at capturing key efficacy outcomes using both traditional methods, and electronic health records or claims data could provide insight into the challenges and opportunities of using real world data to assess outcomes of interest by enabling a comparison between the two. In rare kidney diseases in which patients may be geographically dispersed, decentralized clinical trials and telehealth may facilitate the recruitment and retention of patients and decrease the burden of participation.

Electronic health records data linked with laboratory data and other data sources have been used to explore the safety and efficacy of approved therapies in patients with advanced CKD (i.e., stage 4 and 5 CKD) and those on dialysis, populations that are often excluded from clinical trials. Given the size of the population with stage 4 and 5 CKD and fragmentation of health care among various US health care systems, successful use of such data will likely require collaboration among researchers and the aggregation of real world data from multiple sources. However, these novel approaches should not supplant larger efforts to include these populations in registration trials for drugs, where appropriate, or conduct dedicated RCTs in these populations. As in other rare diseases, potential opportunities exist to conduct nonrandomized, single-arm trials with external controls in rare kidney diseases. Ongoing efforts by various groups to better define the natural history of rare kidney diseases will provide important insight into the diseases and populations in which such a trial design might provide readily interpretable results.

Conclusions

Real world data and real world evidence have the potential to accelerate the development of therapies that provide meaningful benefits to patients. However, there are important considerations, including whether real world data are fit for a particular use and whether a proposed trial design using real world data can provide adequate evidence to support regulatory decision making. There is widespread recognition that there will be a learning curve and that demonstration projects will play a critical role in defining how and when real world data and real world evidence can be used. It is incumbent that the nephrology community participate in the discussions on the use of real world evidence and real world data in drug development, learn from the experiences in other disease areas, and continue to conduct and learn from its own pilot projects.

Disclosures

Dr. Thompson and Dr. Southworth have nothing to disclose.