Real-world evidence and product development: Opportunities, challenges and risk mitigation

Abstract

Real-world evidence (RWE) is derived from real-world data (RWD) sources including electronic health records, claims data, registries (disease, product) and pragmatic clinical trials. The importance of RWE derived from RWD has been once again demonstrated during the coronavirus disease 2019 (COVID-19) pandemic, as it can improve patient care by complementing information obtained from traditional clinical trial programs. Additionally, RWE can generate insights into disease mechanisms, epidemiology, patient flows in and out of healthcare systems, and drivers and barriers to optimal clinical care in real-world settings. Identifying unmet medical needs is crucial as it often can inform which investigational new drugs enter clinical trial testing, and RWE studies from hospital settings have contributed substantial progress here. RWE can also optimize the design of clinical studies, inform benefit risk assessments and use networks of pragmatic studies to help with clinical trial feasibilities and eventual trial initiation. The challenges of RWD include data quality, reproducibility and accuracy which may affect validity. RWD and RWE must be fit for purpose and one must be cognizant of inherent biases.

Introduction

Healthcare costs are rapidly increasing in many settings worldwide. For example, in 2015, the healthcare-related expenditure in the USA amounted to 17.8% of its gross domestic product (GDP). Projections forecast that by 2030 healthcare-related expenditure will amount to 25% of its GDP [1]. This growing financial burden within the healthcare sector is not sustainable, and funders will thus increasingly be required to identify and prioritize where there is more value to be gained. The barriers to establishing new clinical trials are well known, leaving funders reliant on real-world studies to inform cost-effectiveness, patient outcomes, and comparative effectiveness. The establishment of the standards pertaining to and requirements for health technology assessments (HTA) have further promoted the use of real-world evidence (RWE) and real-world data (RWD) to demonstrate safety, efficacy, and cost-effectiveness in real-world settings [1].

RWE is a term used to describe funding from studies based on RWD collected outside traditional clinical trial programs [2]. RWD consist of information collected during routine clinical practice. Sources include pragmatic clinical trials, electronic health records, claims databases, and registries. The RWE results of the analysis of RWD and, provided it is appropriately analyzed, can support and even extend the information provided by typical clinical trial programs. In line with the 21st Century Cures Act and the growing relevance of RWD, the U.S. Food and Drug Administration (FDA) has recently issued a draft guidance for submission of RWD for assessment of investigational new drugs, and new drug applications [3, 4]. This draft guideline will help with the standardization of applications containing RWE for market authorization; it heralds a new era for the optimal use of RWD and RWE. The aim of this paper is to describe how RWE can enhance product development. We also outline the limitations of RWE and suggest strategies to mitigate them.

How does RWE enhance product development?

Although considered as the gold standard, randomized controlled trials (RCTs) have limitations preventing their generalizability to real-world practice. Because of specific inclusion and exclusion criteria, RCTs do not account for the broader population managed in routine clinical practice and its specificities, e.g. vulnerable populations, ethnic differences, comorbid conditions, concomitant drugs, and differences in lifestyles [5]. RCTs are often of limited duration and unable to assess long-term safety and effectiveness and the regular follow-up and close monitoring in most clinical trials do not reflect routine clinical practice. It is clear therefore that RWE would complement traditional clinical trial data, especially in the assessment of safety and efficacy in real-world settings.

Digital health data (often from devices such as digital watches) on top of patient-generated data (mostly generated from electronic health records) are increasingly being used to optimize research and development programs [6]. New technologies are key to address gaps, especially when it comes to personal devices such as mobile devices. Mobiles apps are being used to educate patients about their disease, reinforce treatment prescription and assess compliance and experience [7]. The sheer volume of RWD emerging from such devices is driving the improved understanding of disease processes, therapeutic intervention points and therefore has the potential to enrich clinical studies, reduce time to market and reduce the cost of clinical trial programs [1]. Information provided by RWD allow researchers to develop hypotheses and further investigate clinical research questions such as the burden of the disease or clinical predictions [8]. Importantly, RWE may increase technical and regulatory success [9], thus serving as an important means for pharmaceutical companies to reduce financial risks associated with investing in costly research and development programs. HTA agencies are exploring the use of RWE to contribute to the benefit-risk assessment of drugs [6, 10].

The different ways by which RWE can contribute to phase I–IV trials are described below.

Early discovery and phase I

RWE can establish areas of focus based on unmet medical needs identified in real-world settings, the natural history and burden of an illness, as well as assess risk factors for the disease progression [11]. This can help prioritization of early discovery targets and determine which investigational drugs enter phase I.

Phases II and III

RWE can be used to develop surrogate biomarkers for clinical trials [9] thus shortening the duration of studies while not compromising the assessment of efficacy and safety.

RWE can also play a role in the study design at the time of hypothesis generation or during the assessment of the clinical trial feasibility when the target study population, the duration of the study, and the participating countries need to be defined. By pooling information (incidence, prevalence, comorbidities, mortality, demographics and special patient groups, outcomes, risk factors, treatment options, severity and medical history of a disease), RWD not only optimize patient recruitment via the identification of appropriate patients, but it can also be used to inform sample size calculations [11]. For example, Martina et al. used clinical trial simulations based on RCTs and RWE to demonstrate that the sample size of phase III studies could be reduced by 40% [12].

While the use of clinical trial networks to conduct traditional clinical trials [13] may reduce recruitment times and cost when compared to pragmatic trials, more is still needed. In some cases, historical controls from RWE studies can even be employed to negate the need for controls and thus reduce the number of patients enrolled, ultimately reducing the cost of the clinical trial. The FDA has granted approvals for products based on RWE serving as controls (through historical response rates drawn from chart reviews, expanded access, and other practice settings) [14].

Registration [15]

RWE allows regulators to make ongoing evidence-based decisions in approving and monitoring drugs [16]. Depending on the characteristics and the quality of the RWD used, the FDA considers RWE as scientifically valid for regulatory decision-making and speeding up approvals. The Blincyto study (blinatumomab) is an example of RWE optimizing the registration of a product. Blinatumomab was initially approved by the FDA under accelerated approval with a control arm made of historical data from 694 comparable patients extracted from over 2000 patient records in the European Union and the USA. A further study in a randomized controlled trial was required by the FDA to verify the clinical benefit [3]. The registration of medical devices also serves as an example of the use of RWE, of sufficient quality, to inform or support a regulatory decision.

Phase IV

RWE can inform the long-term safety of registered drugs [17]. Further safety signals or rare adverse effects can be determined thanks to a larger number of patients using the drug for a long duration in real-world settings, as compared to the smaller number of patients using the drug during the clinical trial. This can critically avoid launching an expensive complementary trial. Pragmatic studies can also help determine safety and efficacy in special groups in real-world settings. For example, natalizumab was registered for multiple sclerosis in Japan on the premise that a real-world study is conducted to assess its safety, efficacy and tolerability in clinical practice [18].

Furthermore, RWE has the potential to support labelling changes for an approved product, especially with regards to adding or modifying an indication, changing a dose regimen, use in new populations, comparative effectiveness data, additional safety information, use of drugs in populations with multiple comorbidities and concomitant treatment, etc [3, 4]. Such approaches help to reduce the costs of drugs and increase the return on investment for pharmaceutical companies, allowing for profits to be redirected into research and development efforts.

Real-world pharmacoeconomic studies can also be used to motivate for inclusion in drug formularies and better market access. Since comparative values and cost effectiveness can be evaluated based on RWE, funders are able to decide on drugs to fund and the level of funding. RWD can for instance provide the readmission rates, which could be used to compare products used within hospital settings. Patients often receive several treatment lines and varying devices, especially in oncology and orthopedics, respectively, and costs are often high. Quality of life information and patient satisfaction are important in such settings as these can provide evidence that a treatment is worthy of reimbursement or not. The digitalization of healthcare allows rapid reporting and collection of patient-reported outcomes which could further inform benefit risk assessments [19].

The assessment of medical devices in particular, is associated with varying challenges and gaps. Some of these gaps can now be filled by RWD, especially postmarketing data, electronic health records, registries and billing information [20]. The FDA published a guidance document on the use of RWE to support regulatory decision-making for medical devices indicating the direction it is taking [21]. Fig. 1 outlines how RWE may be used to optimize a product life cycle.

Fig. 1.

The value of real-world evidence for product life cycle management

The use of RWE during a sanitary crisis — The example of the SARS-CoV-2 pandemic/COVID-19

With the emergence and development of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, different strategies have become necessary to manage the outbreak and tackle diagnostics, treatment, and vaccines as quickly as possible. RWE is playing a pivotal role in the generation and collection of data, which helps with the understanding of the disease, its mechanism and epidemiology, but challenges remain.

The FDA has directly engaged with coronavirus disease 2019 (COVID-19) evidence accelerator, a multi-stakeholder project split into two initiatives: the COVID-19 diagnostics evidence accelerator focusing on advancing the development of diagnostics, and the COVID-19 therapeutics evidence accelerator where findings on critical questions are shared. The COVID-19 evidence accelerator consists of weekly virtual meetings (one for each initiative) where experts from the FDA, companies, academic research institutions, device manufacturers, professional societies, payers, and healthcare systems present and discuss information on recent analyses of RWD related to COVID-19 challenges. This collaborative effort allows experts to quickly leverage on the existing expertise [22].

Another example of RWE being key during the crisis is the value brought by observational studies in the research of a treatment. As outlined in Fig. 1, observational studies are important RWD as they provide information in uncontrolled settings generating data on a large and more diverse population. The large number of patients becoming ill in a very short period of time has led to an attempt to repurpose medications for the management of patients with COVID-19 [23]. For instance, the “tocilizumab in patients with severe COVID-19 pneumonia” (TESEO) study was an observational cohort study done in Italy and concluded that tocilizumab might reduce the risk of invasive mechanical ventilation or death in patients with severe COVID-19 pneumonia. The TESEO study provided information which should now be verified by different RCTs such as the ongoing CORIMUNO study [24].

Importantly, cohort studies fill a critical gap in certain populations such as pediatric and adolescent populations for whom data are lacking. For example, through a multinational multicenter cohort study in Europe, Gotzinger et al. have been able to rapidly capture key data in COVID-19 pediatric and adolescent populations from a large number of specialist centers in different countries [25].

What are the challenges with RWD?

RWD and RWE are not without challenges and this has been clearly demonstrated during theCOVID-19 pandemic. RWD must be collected with an intended purpose in line with the patients’ rights, but in order to obtain ethical approval and comply with reviewers’ requests, the transfer of data to third parties may be needed [26].

The context is often critical yet RWD are most often collected on a routine basis related to registries linked to businesses [27]. Additionally, RWE may be limited by confounders and poor data quality, limited accuracy, incompleteness of data and lack of consistency, which even more robust statistical approaches cannot fully addressed [28]. Such data may also be fragmented, and some are provided by personal devices or health-related apps with limited context. This only adds to the difficulty to assess the accuracy and reliability of such RWE and RWD [2].

RWE being obtained by aggregating RWD from different sources, it also raises concerns on data privacy, ownership and sharing [8, 29]. The Office of Health Economics highlighted the heterogeneity in practices in 8 key markets [30]. For instance, in the European Union, data protection is governed by the General Data Protection Regulation (GDPR) as well as national regulations while in the USA it is the role of the Health Insurance Portability and Accountability Act of 1996 (HIPAA).

However, explicit policies warranting that data are processed for specific purposes and with the patient’s consent are still lacking.

Considering the increasing importance of the secondary use of data inherent in RWE, data protection principles are necessary and must be clearly outlined to the patient. According to the GDPR, the patient’s personal data must be “adequate, relevant and limited to what is necessary to the purposes for which they are processed”. This is called data minimization and can be achieved through pseudonymization for instance. This method consists in the deassociation of a subject identity from the personal data being processed [31]. The information allowing subject identification (e.g. the name, email address, social security number, etc) is replaced by a pseudonym (such as randomly generated values). Therefore, the data can “no longer be attributed to a […] subject without the use of additional information”. The collaboration between Google and the Medical Center of the University of Chicago illustrates the complexity around data minimization. By using RWD such as electronic medical records, this collaboration aims at predicting hospitalizations and identifying situations where the patient’s health is declining; however, the hospital was accused of sharing information with Google including date stamps and doctors’ notes and without proper deidentification. Google and the University of Chicago were therefore asked to specify the type and amount of information shared, whether the patients were clearly informed, and whether the patients could opt out of data sharing [32].

Ideally, like studies conducted on human participants, ethical approvals are required when research involves secondary use of data. This would help limit inappropriate and unethical use of data. Even though, as pointed out by Sun et al. [8], the necessity of ethical approvals for some sources of RWD such as retrospective studies has often been questioned, the different principles stated in the Declaration of Helsinki need to be followed. Some specificities also come into play such as the deidentification process, data ownership, details on how long data are kept, etc [30]; however, it must be noted that this would be an administrative burden for researchers. The administrative burden may be limited by the evaluation of these studies by an ethics panel that specifically focuses on RWE studies, without a full sitting of the ethics committee.

Besides, during the SARS-CoV‑2 pandemic in 2020 the conduct of researchers has also been highlighted by the peer-reviewed journal The Lancet given the worldwide impact on the ongoing trials and especially with regards to data sharing. The Lancet made changes in the process to ensure that authors who had access to the data were named in the contributor’s statement. All authors were also required to sign a statement form to confirm they had full access to the data reported in their article [33].

How do we reduce limitations of RWE?

Mechanisms to mitigate limitations of RWE include defining the statistical analysis plan before starting data collection and analysis [34], using appropriate databases to answer the research question, study registration and commitment to publish, matching and adjustment for confounders and sensitivity analysis to test the robustness of the study [35, 36]. Specific policies and guidelines must be established or improved and correctly implemented. They should include systematic guidance on data protection, transparency, quality assurance, patient consent, approval of data collection based on upfront intended use and data ownership. Journals themselves can establish policies to promote appropriate authorship [33]. Research stakeholders with different backgrounds along with policy makers should be brought together to develop standardized practices for the generation and stewardship of quality data. These public and private collaborations can optimize communication and the understanding of RWE. Examples of such collaborations include the Observational Medical Outcomes Partnership in the USA, the Real world Outcomes across the Alzheimer’s Disease spectrum for better care: Multi-modal data Access Platform (ROADMAP) project [37] or the China Real World Evidence Alliance.

Caution and sanity check

The focus on RWE and RWD is relatively new and there is great excitement about its potential value. Although there are many advantages of RWE and RWD, we must keep in mind that the overall benefit risk assessment of interventions must be based on a substantial body of evidence including randomized controlled trials and RWE. RWE may include systematic errors, i.e. bias, and thus these data must be viewed carefully. Furthermore, the conflicts of interest of the researchers and financial contributors must also be considered when reviewing RWE, as duality of interest and financial pressures may affect the choice of which study to conduct and what data to publish.

Conclusion

The RWE established from RWD complements clinical trial programs and constitutes a bridge between the evidence generated in controlled research settings and routine clinical practice. It has the potential to transform and improve clinical trial programs, if used responsibly and in a manner that is fit for purpose. Ultimately, RWE could allow clinical trials to become more cost-effective and speed up the time to market by impacting the licensing, reimbursement, decision-making and uptake of healthcare interventions. Acknowledgement of systematic errors and mechanisms to mitigate bias will improve credibility and validity of real-world evidence. The SARS-CoV‑2 pandemic has been a practical example of how RWE positioned itself as a valuable and powerful tool. The information collected from millions of patients around the world allowed researchers to understand the disease quicker, gather information on how to manage it, and deepen our learning on about how to prevent it from happening again. Once vaccines come to market, if solid evidence is still missing on how to better handle a virus as fulminant as SARS-CoV‑2, would not RWE be the best tool to track and manage it as it continues to reveal itself?

Conflict of interest

P. Naidoo, C. Bouharati, V. Rambiritch, N. Jose, S. Karamchand, R. Chilton, and R. Leisegang declare that they have no competing interests.