Real-World Evidence Should Be Used in Regulatory Decisions About New Pharmaceutical and Medical Device Products for Diabetes

Abstract

Randomized clinical trials (RCTs) are no longer the sole source of data to inform guidelines, regulatory, and policy decisions. Real-world data (RWD), collected from registries, electronic health records, insurance claims, pharmacy records, social media, and sensor outputs from devices form real-world evidence (RWE), which can supplement evidence from RCTs. Benefits of using RWE include less time and cost to produce meaningful data; the ability to capture additional information, including social determinants of health that can impact health outcomes; detection of uncommon adverse events; and the potential to apply machine learning and artificial intelligence to the delivery of health care. Overall, combining data from RCTs and RWE would allow regulators to make ongoing and more evidence-based decisions in approving and monitoring products for diabetes.

Clinical care, regulatory, and policy decisions are based on a continuum of clinical research from intensively monitored randomized clinical trials (RCTs) to real-world observational trial evidence, from tightly controlled, homogeneous populations to broader ones seen in usual clinical practice.¹ In making decisions to approve new pharmaceutical and medical device products, regulatory bodies traditionally rely only on the evidence generated by RCTs.² However, RCTs provide data from a select population and do not necessarily reflect a product’s performance in a broader population. The term “real-world evidence” (RWE) is used by those who develop medical products or who study, deliver, or pay for health care.^3-5 Regulatory bodies have already used RWE to make postmarket decisions, such as labeling changes or product removals, but they have been reluctant in using RWE in premarket reviews. In this article, we suggest that RWE collected from broad, diverse populations can help supplement evidence derived from RCTs and therefore better inform regulatory decisions.⁶

Limitations of RCTS

The main differences between data from RCTs and RWE relate to their distinct aims. RCTs are used to assess efficacy (ie, how a drug or a device performs under well-defined and controlled conditions), notably with a priori defined inclusion and exclusion criteria for the research participants. It is noteworthy that the majority of clinical guidelines are based on evidence from RCTs. With RWE, however, data from the background population provide information on treatment efficacy in a population with fewer predefined restrictions. Some of the differences between how data from RCTs and RWE are utilized are summarized in Table 1.

Table 1.

Examples of Differences in How Data from Conventional Randomized Controlled Trials and Real-World Evidence are Utilized.

Characteristic	RCTs	RWE
Standard of evidence	Gold standard	Complementary to RCTs
Cost	Costly to develop and conduct	Less costly
Patient population	Well-defined within constraints of specific inclusion criteria Results reflect outcomes in limited population	Broader and promotes evaluation of patient populations less often studied in clinical trials Patient data derived from other sources, including insurance claims
Sample size	Limited Requires sample size calculation to be performed in advance	Orders of magnitude larger
Efficacy	Randomized and blinding lead to minimized risk of data bias and confounding	Randomization and blinding may not be feasible Risk of unrecognized data bias and confounding greater
Adverse events	Only more frequently occurring adverse events revealed	Can reveal adverse events with much lower frequency and those requiring longer exposure to occur
Approval or clearance of new medical products	Considered the gold standard necessary for new drug approval, and when feasible for new device approval	Not generally accepted for approving new drugs but can complement RCT findings, accepted for new device indications
Role in diabetes	Define efficacy and provide a preliminary safety profile in a well-defined and controlled population	Allows estimation of more realistic treatment effects of a wide range of diabetes interventions and evaluation of interactions such as social determinants of health and comorbidities
Other issues	May be less useful when strong signals are available from RWE or early-phase trials	Facilitates postmarketing surveillance of adverse events and assessment of the product effectiveness Results may be less credible due when a control group is not included

Source: Adapted from Gyawali et al.⁷

An additional consideration is that persistence with a therapy or device in the real world might be less than in a RCT (given the intense frequency of follow up of participants in most trials). Consequently, RWE may reveal that a therapeutic product may have less effectiveness than suggested by conventional RCTs.⁸

Traditional RCTs may also not be adequate to help understand clinically meaningful pharmacogenomic and pharmacokinetic differences between individuals, which may impact the effectiveness of new products. For example, there is evidence that one-fifth of therapies approved in recent years have differences in drug metabolism or response by race or ethnicity,⁹ yet in the United States, certain racial and ethnic groups that already bear a disproportionate burden of diabetes have been underrepresented in clinical trials of drugs and new technologies.^10,11 Ultimately, if more comprehensive data are obtained for purposes of guiding drug or device selection, this will both benefit regulators and improve opportunities for precision medicine.

Real-World Evidence

In FDA’s 2018 strategic framework announcement,¹² RWE is defined as the clinical evidence regarding the usage and potential benefits or risks of a medical product derived from analysis of real-world data (RWD). The guidance describes RWD as “data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources. Examples of RWD include data derived from electronic health records (EHRs), claims and billing data, data from product and disease registries, patient-generated data including in home-use settings, and data gathered from other sources that can inform on health status, such as mobile devices. RWD sources (eg, registries, collections of EHRs, and administrative and health care claims databases) can be used as data collection and analysis infrastructure to support many types of trial designs, including, but not limited to, randomized trials, such as large simple trials, pragmatic clinical trials, and observational studies (prospective and/or retrospective).”

RWE and RWD are often thought to refer to evidence produced in nonrandomized, observational trials, but FDA’s framework indicates that these terms can also describe randomized trials in the clinical trial setting. These trials are typically very large RCTs and are referred to as large simple trials (LST),¹³ pragmatic trials, or more recently “Efficacy-to-Effectiveness (E2E) Clinical Trials.”^14-16 One of the first examples of the LST was the Salk Vaccine Trial, which led not only to regulatory approval of the vaccine but to a massive nationwide response to a public health crisis.¹⁷ LSTs have been used to support regulatory approval¹⁸ and are not uncommonly used to evaluate diabetes therapies.^19,20

Although RWE, especially from non-RTC studies, has advantages over evidence from conventional RCTs, including lower costs, less time to obtain data, and fewer restrictions in population inclusion, some consider RWE to be less reliable.²¹ RWE studies are subject to design flaws, including unrecognized bias (due to unequal distribution of confounding factors), incomplete datasets, classification errors, and record linkage errors.²² A nonrandomized RWE study is more likely than a RCT to contain unrecognized bias, which can lead to an inappropriate approval or inappropriate new indication of a product if the RWE study is used without RCTs. Due to these potential flaws, a consensus on standard methods for RWE studies is needed to improve the quality and confidence of RWE studies.^23,24

Overall, robust nonrandomized RWE studies, whether conducted in broad populations or in specific intended-target populations (ie, for postmarketing studies evaluating safety, effectiveness, or economic analyses), can complement the evidence generated by RCTs.²⁵ RWE can provide insights into safety, effectiveness, and resource utilization for both a specific intervention and a comparison of multiple alternate interventions. This information can be important to regulators, public health planners, and payers to estimate the effect of a product or intervention on a large population.

The 21st Century Cures Act

The 21st Century Cures Act of 2016 (Cures Act), signed into law on December 13, 2016, is designed to accelerate the discovery, development, and delivery of new cures and treatments for disease. As part of the act, there was authorization to include $500 million over 9 years to help the FDA cover the cost of implementing the changes in the law.²⁶ Among its goals, the act intends to “modernize” clinical trials, including the means by which safety and efficacy data are accumulated and analyzed.²⁷ This would be achieved by the specification of new policies created by the FDA for regulating drugs and biologics.²⁸

On December 6, 2018, FDA launched a new program, the Real-World Evidence Program, to promote the use of RWE as part of its regulatory decision making processes for drugs and biologics.¹² In the document, an introductory scope of the program, which was established by the Cures Act, mentions specifically that this program is intended to support the approval of new indications for an already approved drug or biologic or to help support postapproval studies. The framework document contains: (1) definitions of both RWE and RWD, which is the information that is analyzed to become RWE; (2) current uses of RWD for evidence generation in safety and effectiveness regulatory studies and nonregulatory studies; (3) examples of trials using RWE for nonregulatory purposes to assess comparative effectiveness of treatment regimens; and (4) plans for data standards. The framework document explicitly excludes devices but refers to its corresponding device guidance.²⁹

FDA Regulation of Drugs and Devices Using Real-World Evidence

Drugs

The use of RWE in general is gaining momentum at the Center for Drug Evaluation and Research (CDER). Director Janet Woodcock stated in her 2017 congressional testimony “that real-world evidence is applicable across all phases of drug development. Real-world evidence can help support new indications for existing drugs, because those drugs then would already be on the market, and real-world evidence can also help demonstrate how a drug works in populations that weren’t studied in the trial or relative to another drug not included in the study.”³⁰

Multiple ongoing randomized and observational studies are producing RWE in studying the prevention and care of diabetes-related outcomes. An example is a retrospective, observational study by Sanofi (2018) that compared clinical outcomes in insulin-naïve adults with type 2 diabetes initiating insulin glargine or Insulin Degludec in a real-world clinical setting. Improvements in glycemic control, hypoglycemia outcomes, and discontinuation rates were similar to those of the first completed RCT of insulin glargine versus insulin degludec.³¹ This kind of RWE can therefore serve as a complement to evidence from RCTs for evaluating drug products under conditions that reflect the use of treatments in actual clinical practice and in more diverse patient populations.

As another example, the use of RWE from randomized pragmatic trials has been and will continue to be used to evaluate major adverse cardiac events (MACE) to address both safety and benefit of diabetes therapies. MACE can be objectively assessed even in nonblinded circumstances.³²

While RWE trials cannot completely replace RCTs in assessing the safety and efficacy of new and approved drugs, RWE can certainly augment and expedite efforts in clinical research and approving drugs and devices for diabetes. It is not foreseeable that nonrandomized RWE will serve as well-controlled clinical trials required for drug approval purposes,³³ but large, conventional RTCs that have some degree of pragmatism are already being used for important regulatory purposes.^34,35

Device regulation has developed under a different set of rules. For new or high-risk devices, the FDA has to ensure that a device is safe and effective before approving it to enter the market. For moderate risk devices, such as glucose meters or insulin pumps, the FDA has to ensure that the device is as safe and effective as those already on the market.

Devices

The evidentiary standard set by regulation for devices is more flexible and allows the FDA to use the gold standard of RCTs, as well as partially controlled studies, studies and objective trials without matched controls, and reports of significant experience with a marketed device (ie, real-world evidence). Specifically, the FDA states that “valid scientific evidence is evidence from well-controlled investigations, partially controlled studies, studies and objective trials without matched controls, well-documented case histories conducted by qualified experts, and reports of significant human experience with a marketed device, from which it can fairly and responsibly be concluded by qualified experts that there is a reasonable assurance of the safety and effectiveness of a device under its conditions of use.”³⁶

Given issues previously discussed with RWE and not well controlled trials, the FDA will typically default to requesting RTCs when feasible. On July 27, 2016, the FDA issued a draft guidance on the use of RWE to support regulatory decision making, which was finalized on August 31, 2017.²⁹ This guidance clarified how manufacturers could use RWE to expand of the indications for use of devices that are already on the market. In fact, at about the time that the draft guidance was issued, the FDA also held an advisory panel meeting to determine whether there was enough evidence to change the intended use of the Dexcom G5 continuous glucose sensors from an adjunctive device that complemented fingerstick measurement of blood glucose levels to a device that replaced fingersticks.³⁷ At that time, a majority of the panel recommended that there was reasonable assurance of safety and effectiveness of the device as a replacement for fingersticks based on clinical point and trend accuracy data and reports of significant human experience. As result, the new intended use was approved on December 20, 2016.³⁸

Although this is a good example of the FDA’s willingness to use RWE, it also highlights some of the issues associated with the use of RWE. The first continuous glucose monitoring system (CGM) as adjunctive devices were first approved by the FDA in 2000. It was reported to the FDA advisory panel that most people with diabetes using CGM between 2000 and 2016 had at some point used the sensors as replacements for fingerstick measurements, but because that use was off-label and not recorded, there were no formal data that experts or the FDA could use to analyze. Therefore, the FDA panel and reviewers were left only with anecdotal accounts.

FDA Software Precertification Program

FDA has also announced plans to create a Software Precertification Program. This program is envisioned as a voluntary pathway to assess the safety and effectiveness of software technologies. The program’s goal is to provide more streamlined and efficient regulatory oversight of software-based medical devices from manufacturers who have demonstrated a robust culture of quality and organizational excellence and are committed to monitoring real-world performance.³⁹ Details of how real-world evidence will be selected, collected, and analyzed are pending.⁴⁰ This program will represent another instance in which FDA is using RWE to support regulatory decisions.

Conclusion

Growing appreciation of the value of RWE to complement evidence from RCTs presents opportunities for facilitating the regulatory approval processes for new drug and device therapies and diagnostics. In the United States, the value of RWE is recognized increasingly by the FDA. In addition, it is likely that real-world evidence can also be of value to public health planners and payers to estimate the effect of a product or intervention on a large population. Key challenges for advancing the use of RWE include improving data collection and data quality, and improving the means of analyzing relevant data to mitigate possible biases. These efforts will help to experts and regulatory bodies to rely on RWE to enhance timely decision making for the benefit of all stakeholders involved in diabetes care.