Real-World Evidence: Promise and Peril For Medical Product Evaluation

Abstract

Randomized controlled trials have long been the gold standard for evidence in medical product evaluation, but there is growing support for the use of real-world evidence (RWE). The authors review the benefits and limitations of RWE and discuss the implications for P&T committees.

INTRODUCTION

The development and application of evidence to guide clinical decision-making underlie the promise of medicine to improve human health. For decades, the gold standard for evidence generation in medical product evaluation has been the randomized clinical trial (RCT). However, there is growing interest in the role of real-world data for evidence generation. In this manuscript, we describe what real-world evidence (RWE) means and why it has become increasingly important in the context of evolving evidentiary standards. We first review the current challenges facing those who conduct traditional RCTs. Next, we describe the benefits of RWE, but also its limitations, by discussing different data sources that may represent real-world data and, in turn, contribute to RWE medical product evaluations. Finally, we describe what RWE means for P&T committees and offer thoughts on how P&T committees can become engaged in proactive medical product evaluation.

CHALLENGES TO CONDUCTING RCTS

The past several decades have witnessed impressive developments in medical products and therapeutics—small-molecule drugs, biologics, and medical devices—which have in turn improved health and outcomes for patients. Traditionally, data for these products have been generated through RCTs, the decades-long gold standard for demonstrating safety and efficacy because of their ability to mitigate bias and control for potential confounders. RCTs have well-known limitations that may limit their generalizability to real-world clinical practice. These limitations include conduct in highly selected populations and specialized environments that require intensive monitoring to ensure adherence to study protocol, neither of which may represent everyday clinical practice.¹

While these limitations to traditional RCTs are well-known, the accelerating pace of medical product innovation has been accompanied by additional challenges, many of which relate to their escalating costs. The National Academy of Medicine has concluded that phase 3 trials “have become extraordinarily expensive.”² Estimates of the cost for bringing a new drug to market range from $648 million³ to $2.6 billion,⁴ and the cost of developing drugs has risen approximately 8.5% beyond inflation in recent years mainly because of increasing trial costs.⁴ Cost drivers include: an increasing administrative burden with more complex clinical trial protocols that use multiple assessments, increased responsibility by institutional review boards because of lack of clarity about oversight mechanisms, and inefficient clinical trial monitoring.⁵ There has also been an increased focus on chronic disease drug development in recent years,⁵ which requires longer follow-up time and results in greater trial costs.⁶ With the success of prior therapies in some fields such as cardiovascular medicine, declining event rates have led to the requirement for even larger study populations.⁷ These factors related to rising costs have been exacerbated by a decrease in the rate of investment growth in medical research⁸ and waning site and patient participation.⁹ Therefore, the higher costs, complexity, and longer trial length have led to concerns that RCTs may be inadequate to keep pace with the need for evidence.⁹ Coupled with high failure rates, these factors may make traditional RCTs prohibitive in some areas.¹⁰

EXPEDITING DRUG AND DEVICE APPROVALS

Simultaneously with the challenges in generating data through RCTs, the Food and Drug Administration (FDA) is shifting its paradigm toward a “life-cycle” regulatory approach, which means product evaluations based on integration of both pre-market and post-market evidence.¹¹ Nearly two-thirds of the first-in-class novel therapeutic medications approved between 2005 and 2012 received FDA priority review,¹² which reduces the FDA’s review time to six months from the standard 10 months while directing additional attention and resources to the review.¹³ The FDA is increasingly allowing therapies to come to market on the basis of fewer pre-market clinical trials, or trials that are shorter or based on surrogate measures of disease, on the premise that probable benefits outweigh risks and that uncertainty remaining at the time of approval can be addressed by more thorough post-market evaluation. Multiple pathways have existed for decades to expedite drug approval,¹⁴ and the 2012 FDA Safety and Innovation Act created an additional one, the “breakthrough therapies designation.” This FDA drug review pathway allows approval if preliminary clinical evidence indicates the drug may show substantial improvement over existing technologies.¹⁴ The “breakthrough” designation and other expedited approval pathways have been utilized increasingly in recent years.¹⁵ Through September 30, 2017, the FDA had granted 191 of 500 requests (38.2%) for “breakthrough” designation and denied 244 (48.8%), while 65 (13%) had been withdrawn.¹⁶

Expedited drug approval pathways also often reduce the quantity of evidence necessary for FDA approval,¹⁴ such as enabling enrollment of smaller study populations.¹⁷ Of the first-in-class drug approvals between 2005 and 2012, 34% were on the basis of a single pivotal trial and only 21% used an active comparator.¹² Some expedited pathways also explicitly allow reliance on surrogate measures and biomarkers, instead of clinical outcomes.^14,18 A surrogate measure is “… an endpoint that is used in clinical trials as a substitute for a direct measure of how a patient feels, functions, or survives. A surrogate endpoint does not measure the clinical benefit of primary interest in and of itself, but epidemiologic, therapeutic, pathophysiologic, or other scientific evidence.”¹⁹ Therefore, preliminary approval data relying on surrogate measures often require verification in post-marketing studies.²⁰

Although post-marketing studies have become increasingly important aspects of new drug evaluation, a study examining drugs approved between 2005 and 2012 that had been based on either a single trial or were primarily focused on surrogate measures of disease identified no post-approval studies for 35% of approved indications after a median 5.5 years of follow-up.²¹ There were few published, randomized, controlled, double-blind studies demonstrating superior efficacy based on clinical outcomes. Similarly, 36 of 54 cancer drug approvals from 2008–2012 (67%) were made on the basis of trials primarily focused on surrogate measures; but after a mean 4.4 years of follow-up, 86% of the approved drugs had failed to show gains in overall survival in clinical trials or had not been subsequently tested.²² Recent data on 22 drugs granted accelerated approval for 24 indications between 2009 and 2013 showed that only half of confirmatory studies were completed after a minimum of three years of follow-up.²³ Clinical benefit remained unconfirmed after five years for eight of the indications. Thus, to summarize, although post-approval study data are gaining significance in providing information about drug safety and effectiveness, these studies are often not completed or do not demonstrate clinical benefits when trials are done. Lack of or delays in post-approval study completion may stem from multiple factors, and increased FDA transparency²⁴ about actions taken—or reasons for a lack of action—in response to these studies would be helpful. When post-approval studies show a lack of safety or effectiveness or are inconsistent with earlier data, those data must be integrated into a comprehensive benefit–risk assessment of the drug and, in some cases, may lead to agency action to withdraw market approval or require a more definitive study. Paradoxically, although expedited approval pathways rely on post-approval data, once a drug is approved, enrolling patients in studies can become more difficult because patients are less incentivized to participate in clinical studies since they can receive the drug without participating in a trial.¹⁴ When such studies are completed, the results must be reported at ClinicalTrials.gov and may also be published in the peer-reviewed literature.²⁵

FDA medical device regulation has undergone a similar transformation of enabling expedited approvals with increasing reliance on post-marketing data. The FDA introduced an expedited access pathway for medical device review in 2015.²⁶ This pathway was created under the premise that it may be appropriate to accept greater uncertainty in order to expedite availability of devices to patients, with greater dependence on post-marketing studies.²⁷ The 21^st Century Cures Act, signed into law on December 13, 2016, created a Breakthrough Devices Program^28,29 analogous to the preexisting one for drugs. However, post-approval commitments for medical device clinical studies are often not fulfilled;³⁰ within three to five years after medical device approval, only 13% of post-marketing clinical studies are completed.³¹

REAL-WORLD EVIDENCE

Given the parallel trends of expediting drug and device approval with somewhat less reliance on pre-market data and the challenges of conducting traditional RCTs, there has been a move toward novel ways of generating evidence. In recent years, the volume and complexity of electronic health care real-world data, which can in turn be used for RWE medical product evaluations, has grown exponentially. Although observational data have been used to study medical products for decades, the granularity and complexity of electronic health record (EHR) data, coupled with EHRs’ linkage to longitudinal data for outcome ascertainment, and advances in data science analytics, together offer substantial promise to leverage these data for medical product evaluation, informing regulatory decision-making, P&T coverage decisions, and ultimately clinical practice.

Defining Real-World Evidence

RWE is defined in Section 3022 of the 21^st Century Cures Act (which creates Section 505F of the Federal Food, Drug, and Cosmetic Act) as “data regarding the usage, or the potential benefits or risks, of a drug derived from sources other than randomized clinical trials.”^28,32 Although the law appears to suggest that randomized data do not make up RWE, FDA authors have clearly stated that this is not the case, writing that it is “incorrect to contrast the term ‘real-world evidence’ with the use of randomization in a manner that implies that they are disparate or even incompatible concepts.”¹ More recently, FDA authors clearly said that it was “not the intent of Congress” that single-intervention clinical trials be used to generate RWE.³³

The FDA has defined RWE as “information on health care that is derived from multiple sources outside typical clinical research settings, including EHRs, claims and billing data, product and disease registries, and data gathered through personal devices and health applications.”¹ In August 2017, the agency finalized a guidance entitled “Use of Real-World Evidence to Support Regulatory Decision Making for Medical Devices.”³⁴ This guidance made the key point that it does “not change FDA’s evidentiary standards for regulatory decision-making.”³⁴

Potential of RWE

RWE can efficiently provide relevant clinical data in populations representative of those seen in clinical practice. Clinical trials are often conducted in populations who are younger, more often male, and less racially and ethnically diverse than those seen in clinical practice.^35–38 Everyday practice also includes other important clinical factors such as patient adherence, tolerance, comorbidities, concomitant treatments, study location, and environment, all of which are more standardized and controlled in RCTs.¹⁰ RWE may also better account for complexities—such as the fact that medications need to be tested in combination with other medications during development. Further, the patient populations receiving medical products will evolve over time; RWE can better take these factors into account, since repeating randomized trials may be implausible. Finally, RWE is rooted in clinical practice, where value considerations are becoming increasingly important.^10,39 In some cases, it may be difficult (because of rapid innovation and development cycles) or unethical to randomly assign people to one treatment or another; existing real-world data may offer a reasonable substitute for prospective trial data.²⁷

Additionally, RWE has the capacity to inform regulatory decision-making. Safety signals for new drugs and devices may not emerge until the treatments are available in clinical practice, when a larger number of patients receive them and can be followed for a longer duration than in clinical trials. Efficient detection of adverse events through RWE could inform post-market use, including drug or device label changes. Similarly, RWE may help to support new indications for therapies; a recent example is the FDA approval of additional indications for transcatheter heart valves by using data collected in a post-market valve registry that had been mandated for Medicare coverage.⁴⁰ Therefore, RWE is increasingly being used to inform post-approval safety monitoring and additional indications.

Incorporating the Patient’s Voice

RWE can also better represent patients’ voices and experiences, in part through patient-generated data. Section 3011 of the 21^st Century Cures Act requires the FDA to include a statement on the use of patient experience data in its regulatory decisions.²⁸ Patient experience data was defined in the legislation as “data collected by any persons (including patients, family members and caregivers of patients, patient advocacy organizations, disease research foundations, researchers, and drug manufacturers).”²⁸ The goal of these data is to inform the FDA about the impact of a disease or therapy on patients as well as patient preferences. While incorporating patient input regarding values and preferences will be very valuable and is already done in some RCTs through patient-reported outcomes, these data should be rigorously and systematically collected. Patient advocacy organizations may play an important role in generating and communicating these data, but at least two-thirds of these organizations receive industry funding, which may introduce conflicts of interest.^41,42 Current disclosure practices of most advocacy organizations are limited,⁴² and these groups may receive funding from multiple companies.

Strengths and Limitations of RWE

While RWE clearly has demonstrated significant potential—which will undoubtedly continue to mature and expand—important caveats must be considered when determining if RWE can be relied upon for decision-making. RCTs have been the gold standard for decades primarily because randomization allows for the balancing of measured and unmeasured confounders, thus minimizing bias. The central limitation of using RWE for medical product evaluation is that it ultimately asks us to make inferences and evaluations from observational data, which can sometimes be a precarious endeavor because of the presence of unmeasured confounders.⁴³ There is a need to advance statistical methods for reliable analyses of RWE.⁴⁴ Understanding the various data sources that may comprise RWE is critical; they have different strengths and weaknesses. Used appropriately, RWE can both help check for consistency and also provide different perspectives on important clinical research questions.¹⁰

Ideal Data Source

The ideal data source used to inform health care–related decisions would include a representative sample of patients with the underlying condition—both when considering demographic as well as clinical factors (Table 1). This data source would be planned prospectively and would offer continuously updated longitudinal follow-up for a full array of clinical outcomes, including those related to patient-reported measures of health and well-being. This data source would also have quality control measures in place to ensure data validity. To minimize the data collection burden, such a data source would integrate with existing data systems and require minimal resources to collect data.

Table 1.

Characteristics of an Ideal Data Source to Generate Real-World Evidence

Patient population representative of those with the underlying condition (e.g., demographics, clinical comorbidities) Prospectively planned Continuously updated with minimal resources Longitudinal follow-up Rich clinical data: clinician-entered, patient-reported, and patient-generated Quality control measures in place Integrated within existing data systems

While no data source has these ideal characteristics, studies should be conducted with the most rigorous possible design by investigators without bias and using appropriate analytic approaches through familiarity with the research database. RWE is likely to consist of multiple real-world data sources.⁴⁴ With this in mind, we will discuss different types of RWE.

Clinical Registries

Clinical registries are heterogeneous and numerous. Some of the largest registries have been established by professional societies. For example, within cardiology, the American College of Cardiology’s National Cardiovascular Data Registry includes 14 unique registries encompassing both outpatient and inpatient care, including specific procedures such as implantable cardioverter-defibrillator placement. Registries involve either systematic collection through trained abstractors or automatic capture of standard electronic data elements. Advantages of patient registries include clinically rich and generally consistent data. However, registries may not capture longitudinal data or may require significant effort and resources to do so. Additionally, registries may not necessarily be representative of the general clinical use of a given drug or device, and they may not include the full spectrum of outcomes, such as patient-reported outcome measures. Registries often exist in parallel to existing data collection systems, resulting in some duplication for greater accuracy, and they may require resources such as dedicated, trained staff. Finally, registry data may not be available in a timely manner, and therefore may lag behind clinical practice. Registries are often limited to the specific procedures, diseases, or settings that they are designed to capture.⁴⁵ An example of registry data assisting in evaluation of a medical device is a registry-based analysis of the effectiveness of intra-aortic balloon pumps among patients undergoing high-risk percutaneous coronary intervention, which found significant hospital-level variation without any in-hospital mortality differences.⁴⁶ The value of registries may be shown by the creation of a “Registry of Patient Registries” by the Agency for Healthcare Research and Quality.⁴⁷

Claims Data

Administrative claims data are even more ubiquitous than clinical registries. Examples of claims databases include Medicare or any other insurance payer’s claims data, OptumLabs Data Warehouse (the health services platform for United Healthcare, which includes longitudinal data for more than 100 million people),⁴⁸ and Premier (a health care alliance of more than 3,500 U.S. hospitals and 100,000 providers).⁴⁹ Advantages of claims data include the ease of data collection and abstraction because these data are universally created for billing purposes, the ability to capture longitudinal data (provided that a patient remains within the same insurer’s system), and widely accepted coding standards that can support data consistency.⁴⁵ Claims data can often be integrated with existing data platforms. However, claims data are not collected with the goal of supporting research.¹ Their disadvantages relate to coarseness with a lack of detailed clinical information, including inability to identify the use of a specific drug or device, uncertainty about adequate risk adjustment, and the movement of people from one insurance plan to another. Additionally, claims data may not be able to differentiate comorbidities from complications and may not be complete within a given episode of care.⁴⁵ Claims data also suffer from time lag, since coding is performed after an episode of care. Finally, claims data may be inaccurate;⁵⁰ a study comparing claims to independent study physician adjudication within an observational cardiovascular study of adenosine diphosphate receptor inhibitors found that claims were only modestly accurate in identifying stroke and myocardial infarction and had even more limited accuracy for bleeding events.⁵¹ Claims data are quite accurate for identifying health care utilization, such as hospitalizations.⁵² As an example of claims data assisting in evaluation of a medical device, a study using New York state claims data compared patients receiving laparoscopic sterilization with hysteroscopic sterilization over a nine-year period and showed patients receiving hysteroscopic sterilization were more likely to undergo reoperation.⁵³ Given that the Essure device (Bayer) was the only hysteroscopic sterilization therapy available during the period in question, the study served as an evaluation of that device.

Electronic Health Record Data

EHRs have among the greatest potential to act as a source of RWE. The Health Information Technology for Economic and Clinical Health (HITECH) Act, signed into law as part of the American Recovery and Reinvestment Act of 2009, legislated the adoption of electronic medical records.⁵⁴ Consequently, the past decade has witnessed increasing and now near-universal adoption of EHRs, making them a ubiquitous source of data. Over the past five years, the PCORnet common data model has been developed; it is a distributed research network with data from multiple clinical data research networks (EHR-based) and patient-powered research networks mapping data in a consistent format to a single common data model.⁵⁵ However, as with claims data, EHRs are not designed to support research,¹ and they are infrequently optimized for this purpose. Advantages of EHRs include detailed clinical information, which consists of demographics, diagnoses, narrative text notes, electronic procedure and test reports, laboratory data, vital sign records, medication lists, order/entry, and other items.⁴⁵ If patients remain in the same health care system, EHR data will also be longitudinal. Disadvantages of EHRs include their lack of interoperability and uneven data quality. If patients move between health systems with different EHRs, data will be incomplete. Additionally, while machine learning tools such as natural language processing are increasingly being applied to EHRs, including for purposes such as monitoring pharmaceuticals for adverse events,⁵⁶ the unstructured nature of much of the data often makes EHR-based analyses cumbersome. Additionally, EHRs seldom systematically collect patient-reported outcome measures in an abstractable format,⁵⁷ although this information can sometimes be found in clinician notes. Evolving reimbursement paradigms that are increasingly focused on quality and value of care may incentivize more accurate data entry by providers.¹⁰ An example of EHR data being used for evaluation of a medical device is a study conducted at three institutions using their own EHRs and finding that patients receiving the Thoratec Heartmate II left ventricular assist device were experiencing higher rates of device thrombosis than in the past, and that this adverse event was associated with significant morbidity and mortality.⁵⁸

Pragmatic Clinical Trials

Among the most important promises of new RWE-generating mechanisms is the ability to conduct pragmatic clinical trials, in which research is embedded within routine clinical practice among patients who best represent the population of patients with a disease while maintaining the rigor of randomization.⁵⁹ As traditional trials are focused on demonstrating efficacy, there is concern that risks are underestimated and benefits are overestimated.⁵⁹ The hope is that pragmatic trials can address this through enrollment of larger study populations more representative of people with a given condition, including fewer restrictions on the use of concomitant therapies or on inclusion of patients with other comorbid diseases. Pragmatic clinical trials are also cheaper than traditional RCTs and may be able to obtain data on a larger number of clinical outcomes.⁹ Disadvantages of these trials include their requirement of an infrastructure to facilitate enrollment. There are also concerns related to the data source; with EHRs, as noted above, there may be loss to follow-up and/or incomplete or inaccurate ascertainment of outcomes because EHRs are not designed specifically for research purposes. Such studies will work only for certain endpoints that require limited monitoring.⁵⁹ There may also be a higher occurrence of protocol violations.⁹ Whether pragmatic clinical trials will be acceptable for new pharmaceutical and device approval remains unknown.⁹ An example of a pragmatic clinical trial is the Post-Myocardial Infarction Free Rx Event and Economic Evaluation (MI FREEE), in which insurance-plan sponsors were randomly assigned to full prescription coverage or usual prescription coverage for patients discharged after hospitalization for myocardial infarction to understand if eliminating patient copayments would improve cardiovascular events and adherence. Claims data were used to both identify potential study participants and evaluate clinical trial outcomes, including a primary endpoint of first major vascular event or revascularization.^60,61

Active Surveillance Systems

Active surveillance systems are another important source of RWE that can be used to evaluate medical product safety after approval. An active surveillance system tracks experience with devices and monitors prospectively analyzed data, with triggers set for deviation from expected outcomes.⁶² Advantages of active surveillance systems include their rapidity and minimal marginal costs because they rely on available real-world longitudinal data. Disadvantages include their limitation to tracking products for which relevant data are available and the need for methods and data to match patients receiving various devices. Findings from active surveillance systems also require validation, such as that offered by medical chart review or more formal epidemiologic studies, as there may be residual confounding from covariates not included in the models.⁶² Active surveillance systems require standardized event terms with unique methodology that must be continuously developed and reiterated for use across multiple data sources for different products. An example employing an active surveillance system is use of the DELTA (Data Extraction and Longitudinal Trend Analysis) system in a study conducted within the National Cardiovascular Data Registry’s CathPCI Registry that identified a specific femoral vascular-closure device as associated with a higher risk of vascular complications compared with alternative vascular-closure devices.⁶³

Emerging Technologies

Emerging technologies (digital health) have the potential to contribute RWE in decision-making related to the use of medical products. Rapid technological innovation over the past several years has led to the development of thousands of health-related mobile applications, wearable technologies such as fitness trackers, and sync-able technologies such as digital weight scales and portable electrocardiographic sensors. These technologies are contributing to progressively greater data streams. They generally involve patient-generated data and also offer the prospect of recruiting patients virtually.¹⁰ For example, the Health eHeart Study is a cardiovascular-focused eCohort in which people enroll solely using the Internet and can enter their medical history and lifestyle habits, as well as connecting multiple devices and applications.⁶⁴

Advantages of emerging technologies include the ability to capture detailed and longitudinal information, including patient-generated data and patient-reported outcomes. Given that nearly all of people’s activities occur outside of health care settings,¹⁰ these technologies can provide rich data for a more complete understanding of health. Disadvantages of these technologies include the potential absence of representative populations. These technologies may not be adapted into health care data platforms and may capture limited outcomes. Accuracy and validation of data from these novel technologies are also important; data suggest wearable devices may have up to 20% variation in obtaining accurate step counts,⁶⁵ and blood pressure measurements from an instant blood pressure smartphone application have been shown to be inaccurate.⁶⁶ Patient-reported outcome measures obtained electronically have generally been found to be equivalent to traditional paper-based reports if there are only minor changes.⁶⁷ Emerging technologies also need to navigate infrastructure, legal, and privacy-related concerns. The FDA recently launched a Digital Health Innovation Action Plan, including a Pre-Certification for Software Pilot Program to help set expectations about regulation of digital health products.⁶⁸ The potential of emerging technologies was demonstrated in a recent smartphone application monitoring study that recruited nearly 50,000 participants over an eight-month period.⁶⁹ More than 40,000 participants uploaded data related to activity and sleep patterns through both sensors and patient questionnaires. Social media data may also contribute to this area, such as streams from Twitter and Google, which were shown in 2009 to estimate weekly influenza activity in the United States.⁷⁰

Data Sharing Platforms

Data sharing platforms may also serve as an important resource in an era where regulatory decision-making increasingly relies on RWE. Through a learned intermediary or directly, data generators can share their clinical study data, allowing for investigators and regulators to maximize the value of the clinical trials that have been conducted. Data can be aggregated to allow for a more comprehensive understanding of drug or medical device safety and efficacy. Momentum for data sharing and transparency has increased in recent years. An example is the National Heart, Lung, and Blood Institute (NHLBI) Biologic Specimen and Data Repository Information Coordinating Center, which coordinates activities of the NHLBI biorepository and data repository and has been used to publish 277 articles from 47 clinical trials.⁷¹ Advantages of data-sharing platforms include the low cost of examining detailed clinical data, which can be used to conduct additional studies,⁷² systematic reviews, and comparative effectiveness studies, as well as to validate published results.⁷³ Disadvantages are that data-sharing platforms are limited by data collected within clinical trials, with their attendant limitations such as external generalizability. An example of using data sharing for pharmaceutical evaluation was a re-evaluation of a double-blind, randomized trial funded by Smithkline Beecham and published in 2001 showing paroxetine was an effective treatment for depression in adolescents.⁷⁴ Re-analysis of data by independent investigators, however, showed that paroxetine had no clinical benefit.⁷⁵

RWE and P&T Committees

There will be growing reliance on RWE for health care decision-making in the future, along the spectrum from FDA approval to clinical decisions made between physicians and patients at the bedside. As the FDA increasingly moves toward a life-cycle approach to regulation, this will mean that more continuous evaluation of technologies and treatments will be necessary through post-market data, including assessments by P&T committees. Responding to new and emerging safety information will be particularly important given that the FDA takes a serious safety-related action for nearly one-third of drugs after their approval, including a withdrawal, imposing a boxed warning, or issuing a communication directly to clinicians.⁷⁶ While real-world data sources have more often been used for safety surveillance than for effectiveness evaluations, P&T committees will similarly need to monitor and respond to new and emerging data on medical product effectiveness. Use of post-market data to inform effectiveness evaluations is becoming increasingly important as drugs and devices are approved on more limited evidence, but there may be greater bias in effectiveness evaluations, and these require the ability to critically evaluate RWE and how “close to ideal” the data source may be for the purpose (Table 1), taking into account the strengths and limitations reviewed above. Finally, RWE will increasingly supplement traditional sources of evidence to augment generalizability.

Given the central role of P&T committees in evidence-based decision-making, they will need to evaluate and apply findings from different real-world data sources to inform their formulary development, and they will have to do so more quickly than in the past. Traditional RCTs may become less frequent, supplanted by pragmatic RCTs and evaluations using nonrandomized data; P&T committees will need to consider the totality of this evidence and first evaluate the source of data, methodology, and results. Next, P&T committees should determine if, and how, the results should be considered based on their strength as evidence to inform formulary coverage decisions. When possible, such decisions should be complemented by local experience. Finally, there will have to be more rapid evolution in formulary decisions over time as new evidence becomes available because of the increasing reliance on post-approval data.

P&T committees may also have the opportunity to review internal data generated within the health system over which they have purview, particularly EHR data, which can complement published data from other sources. This will require collaboration with local experts who have both methodological and analytical expertise to consider strengths and limitations. These data have the potential to inform P&T committee decision-making that is gathered from—and more directly relevant to—the unique patient population of the P&T committee’s organization. P&T committees should proactively engage in discussions about medical product evaluation—data sources, methods, and analytic plans—for specific products where additional data would help inform evidence-based determinations; this will ensure that the outcomes observed within their facilities will support their decision-making, addressing the most pertinent and actionable knowledge gaps for their patients.

Once these evaluations have been completed, it is equally important that P&T committees take action. For example, formularies for health plans providing prescription drug coverage to Medicare beneficiaries made statins widely available before and after the 2013 American College of Cardiology/American Heart Association Cholesterol Guideline⁷⁷ recommended their use to reduce cardiovascular risk; however, in response to these same guidelines, which relegated nonstatin therapies to second-line use given their limited clinical benefits, formularies on average increased in restrictiveness, but they could have become more restrictive to promote the use of the most effective drugs for Medicare beneficiaries.⁷⁸ Similarly, there are also opportunities for formularies to change in response to new evidence on drug safety in order to promote safer prescribing, such as by increasing restrictions on opioid medications given the ongoing opioid crisis⁷⁹ and by curtailing access to drugs with safer available alternatives that receive FDA boxed warnings.⁸⁰

CONCLUSION

Now more than ever, we are witnessing the emergence of exciting possibilities for the generation of RWE that can support medical product evaluation to inform patients, clinicians, payers, and policymakers—including P&T committees. RCTs remain the gold standard of evidence, but with the challenges facing the traditional research infrastructure, we are likely to see fewer traditional trials in the future and their replacement with pragmatic trials and other forms of RWE. The evolution of research paradigms will continue to accelerate with the availability of larger volumes of electronic health care data. But depending on the data source, there is a significant need to ensure that analyses are reliable and can serve as the foundation for generalizable knowledge about safety and effectiveness. P&T committees can proactively engage in medical product evaluations to focus on policy questions and take action to ensure the highest-quality care for their patients.