A Review of the Importance and Relevance of Real-World Data and Real-World Evidence

Abstract

Real-world data are routinely collected data associated with patient health status or delivery of health care from sources including patient registries, electronic health records (EHRs), medical claims data, or digital health technologies. Real-world evidence is generated from specified clinical real-world data and includes evidence of the use, benefits, and risks of a medical product. Analysis of real-world data is the basis of real-world evidence to support the use and potential benefits or risks of a medical product. Also, real-world insights are generated when real-world evidence is interpreted or applied by different stakeholders in the healthcare industry to inform the planning of clinical research studies, identifying research questions, relevant patient or disease groups, disease trends over time, and evaluating the commercial viability of a study, which can improve healthcare planning and implementation. The importance of standardizing and regulating the conduct and reporting of real-world studies is highlighted by the increasing number of registered studies. This article aims to review the continued importance and relevance of real-world evidence from real-world studies and data to support and refine interpretations from controlled clinical trials.

Introduction

According to the US Food and Drug Administration (FDA), in medicine, real-world data are routinely collected data associated with patient health status or delivery of health care, which is collected from a variety of sources, including patient registries, electronic health records (EHRs), medical claims data, or digital health technologies [1]. Studies on real-world evidence are a 21^st-century phenomenon, often synonymous with evidence from patient registries [1]. According to the Oxford English Dictionary (OED), the noun “real world” was first documented in the 18^th century and used to distinguish between reality and imagination (theory or dreams) [2]. Between 1966 and 2004, there were 2,852 publications identified in PubMed on real-world evidence, real-world study, or real-world data. In the 1990s, review articles began to recognize that real-world evidence of the safety and efficacy of approved drug treatments could be different from clinical trial data in which study participants were selected based on inclusion criteria that would not commonly be found in clinical practice [3].

In 2004, an early real-world evidence-based study was published by Simons and Krishnan on the costs of lamotrigine treatment to stabilize mood for patients with bipolar disorder, and used longitudinal administrative claims data [4]. This early real-world evidence-based study showed that initiation of lamotrigine was associated with reductions in hospital days, with net cost savings of more than $400 per patient per year [4]. In the past 20 years, between 2004 and 2024, there were 2,852 publications identified in PubMed on real-world evidence, real-world study, or real-world data. In 2016, Oyinlola and colleagues published a systematic review and meta-analysis study to identify the extent of the direct impact of real-world evidence on health and social care systems in England from 2000 [5]. National guidelines or guidance published in England were identified that referenced studies using the Clinical Practice Research Datalink (CPRD), which is a governmental primary care data provider [5]. The study identified 25 guidance documents referencing 43 different CPRD/GPRD studies, published between 2007 and 2016, covering 12 disease areas, with all 43 studies providing evidence of the epidemiology, prevalence, pharmacoepidemiology, pharmacovigilance, and health utilization [5]. Between 2007 and 2016, there was a slow uptake in the use of real-world evidence in clinical and therapeutic government guidelines [5]. However, the authors identified an increasing trend in the use of healthcare systems data to inform clinical practice and indicated that this trend developed as the real-world validity of clinical trial data began to be questioned [5].

A persistent problem is that the adjective “real-world” is also increasingly used for studies that refer to applications and experience within clinical practice, which may be small observational studies without adequate supporting data [6]. Analysis of real-world data is the basis of real-world evidence to support the use and potential benefits or risks of a medical product [1,7]. This article aims to review the continued importance and relevance of real-world evidence from real-world studies and data to support and refine interpretations from controlled clinical trials.

Real-World Data, Real-World Evidence, and Real-World Insights

There is still confusion regarding the use of the term “real-world”. The US Food and Drug Administration (FDA) defines real-world data as being patient health data collected routinely from different sources and qualifies the data sources as including electronic health records (EHRs), claims and billing activity, patient registries, domiciliary patient-generated data, and electronic sources [1]. Real-world evidence is generated from specified clinical real-world data and includes evidence of the use, benefits, and risks of a medical product [1]. Also, real-world insights are generated when real-world evidence is interpreted or applied by different stakeholders in the healthcare industry [1]. Real-world insights inform the planning of clinical research studies, identifying research questions, relevant patient or disease groups, disease trends over time, and evaluating the commercial viability of a study, which can improve healthcare planning, implementation, and business-related decisions [7–9].

How Real-World Data and Data from Controlled Clinical Trials Compare

Controlled clinical trials are conducted in selected patient populations and controlled settings that have strict inclusion and exclusion criteria, study protocols, and methods to quantify treatment effects, and are of limited duration. The advantages of real-world evidence from real-world data, when compared to controlled clinical trials, can best be appreciated in the context of the known limitations of clinical trial data (Table 1) [10,11]. Real-world evidence often addresses the limitations and disadvantages of clinical trial data and provides answers to many of the well-known disadvantages of trials [12]. In the real world, patient populations and situations are diverse, and there will inevitably be limitations in applying the findings from controlled clinical trials of a selected population to the general patient population [11]. Therefore, the advantages of real-world data and real-world evidence are that data are obtained without strict eligibility criteria, and will include patients with comorbidities and concomitant medications [12]. Real-world studies can be done more quickly and are more cost-effective as there is no need for patient recruitment and enrolment strategies [12]. An important consideration is that real-world studies can also include patients considered to be at too high risk for controlled clinical trials, such as children and pregnant women [12]. The size restriction of clinical trial populations can be overcome in the real world to allow larger studies with subpopulation analysis that is facilitated by easier access to real-world study data [12]. Therefore, although controlled clinical trials are the gold standard for safety and efficacy evaluation of drugs and medical devices, it is clear that real-world evidence should be regarded as complementing the findings of controlled clinical trials (Table 1) [10,11].

Table 1.

Comparison of real-world data and data from controlled clinical trials and studies [10,11].

	Real-world data	Data from controlled clinical trials and studies
Aims	Effectiveness/response	Efficacy
Setting	Real-world clinical practice	Controlled research
Patient inclusion criteria	No strict criteria for patient inclusion	Strict criteria for patient inclusion
Drivers of data	Patient-centered	Investigator-centered
Medication interactions and comorbidities	Real-world clinical practice	Only included according to the study protocol
Role of the physician	Multiple physicians, as decided by the patient	Designated investigator
Comparator	Patient need, real-world and variable treatments, as determined by the market and physician	Placebo or standard care
Treatment	Variable treatments, as determined by the market and physician	Fixed, according to the study protocol
Response monitoring	Variable	Continuous through the study
Patient follow-up	Determined by real-world clinical practice	Variable, according to the study protocol

The benefits of real-world evidence to key health stakeholders include patients, healthcare providers, pharmaceutical and medical device companies, regulatory safety and pharmacovigilance monitoring, and payers (Table 2) [10,11]. The generation of real-world evidence depends on collecting data in real-world clinical settings, using several sources, including electronic health records (EHRs) and databases, claims databases, patent registries, and health-related data from electronic devices, social media, and patient support and health groups [7,11].

Table 2.

Benefits of real-world evidence to key health stakeholders [10,11].

Stakeholders	Benefits of real-world evidence
Patients	Patient access to social media, the internet, and patient support groups has raised patient awareness of clinical trial data and real-world evidence. Patient awareness has encouraged reporting of safety concerns, the effects of comorbidities, and long-term outcomes, and participation in real-world studies
Healthcare providers	Real-world evidence from local and national electronic health record (EHR) analysis can provide support for providers to negotiate with drug and medical device manufacturers to retain or replace treatments, evaluate cost benefits, and seek costs from manufacturers when real-world patient outcomes do not match controlled clinical trial data
Pharmaceutical and medical device companies	Pharmaceutical and medical device companies now use real-world evidence throughout the product lifecycle. Real-world data can improve standards and procedures for the conduct of clinical trials, provide insights into the impact of drugs and medical devices in a target patient population, inform trial design, improve clinical guidelines, facilitate financial and reimbursement decisions, support regulatory decisions, and promote new or expanded uses for marketed products
Regulatory safety and pharmacovigilance monitoring	Regulatory agencies use real-world evidence to monitor the safety of marketed products using traditional pharmacovigilance methods and new digital systems. For example, the US Food and Drug Administration (FDA) Sentinel Initiative infrastructure supports several external projects from a variety of stakeholders as a global learning health system [20]
Payers	Healthcare funders (payers) in the US use claims data to improve the affordability of new products and implement outcomes-based contracts with providers and prescribers. In the UK, the National Institute for Health and Care Excellence (NICE) includes health technology assessments (HTAs) to compare patterns of treatment and inform pricing and reimbursement decisions [40]

Real-World Data from Hospital Electronic Health Records

A recent editorial in the New England Journal of Medicine has highlighted the limitations and the potential for using hospital electronic health records (EHRs) as a source of real-world data [13]. Robust real-world data from EHRs requires improving real-world data capture at the point of care and facilitating access to longitudinal individual health data, supported by streamlining clinical research regulatory practices [13,14]. Almost all hospitals in the US have adopted EHRs, which document individual admissions and care episodes to facilitate healthcare payments for that care [13]. Although EHRs were not developed to support clinical research, their role as a resource for real-world clinical data could be improved if the data were more comprehensive and important data and outcomes were accurately recorded [14]. Currently, data from EHRs may not adequately establish risk factors and causes of disease or treatment outcomes due to confounding or inadequate documentation and due to expensive and time-consuming manual data collection [15].

There have been recent approaches to improving the quality and availability of real-world data from EHRs at the US federal level, particularly to improve research on chronic disease and improve clinical outcomes [13,15]. Firstly, because data generated during the provision of routine care are frequently inaccurate or incomplete, point of care data capture could be standardized [13,15]. Second, even when hospital EHRs contain high-quality real-world data, access to detailed longitudinal data may be limited when patients receive care at multiple hospitals or care centers [13,15]. Thirdly, there are gaps in clinical data from EHRs that may not be available from nursing homes and some other health facilities [13,15]. Real-world data from clinical centers will likely rely on improved EHRs, claims data, and improved mortality data, which currently have limitations [13,15]. A solution to these limitations could include the provision of incentives to improve data quality, including by making participation in clinical evidence generation a prerequisite for funding [13,15]. Artificial intelligence (AI) systems could also reduce the burden of data capture, although the use of AI systems remains to be tested in EHRs [13]. Recently, the US Department of Health and Human Services (HHS) facilitated the launch of a network for the exchange of health data [16]. The Trusted Exchange Framework and Common Agreement (TEFCA) will allow participating clinicians, patients, and funders to exchange health information with the potential for research-related exchange of real-world data [16]. Israel has implemented a national patient discharge database, which has provided real-world data on outcomes following vaccination, including the documentation of rare vaccine-associated adverse events [17]. Importantly, improvements in real-world data infrastructure also require regulatory support for these data to be evaluated in clinical research [13,15]. In 2018, the US Food and Drug Administration (FDA) issued guidance documents on the use of real-world data and real-world evidence in regulatory applications [18,19]. However, it is still unclear how this guidance can be implemented at a time of limited staffing and funding [18,19].

Real-World Data from Claims Databases

Most claims databases are in the US, with Medicare and Medicaid being the prime sources of health claims data [7]. In the US, healthcare claims database studies are usually longitudinal, retrospective, and cross-sectional analytical studies of data obtained from healthcare and health administrative databases [7]. The aim is to analyse healthcare resource utilization (HCRU) and healthcare costs from the data entered by pharmacies and health insurers [7]. Healthcare claims database studies assess the long-term effectiveness of health interventions in real-world settings, including inpatient, outpatient, emergency room, clinic visits, diagnostic procedures, surgical procedures, pharmacy services, hospitalizations, and length of hospital stay [7].

Recently, the US FDA created the Sentinel System, following the recommendation from the FDA Amendments Act (2007) for a system to monitor the risks associated with data from varied sources regarding the use of drug and biologic products, and to develop governance and act as a national resource [20]. Currently, the infrastructure of the Sentinel System supports several non-FDA studies and informs stakeholders, including federal agencies, international regulatory bodies, and academic researchers [20]. The Sentinel System can be regarded as a learning health system that has the potential to develop into a global learning health system to improve safety and efficacy assessments for drug products and devices [20].

Real-World Data from Patient Registries

Patient registries are organized systems that prospectively collect and analyse observational data on defined patient populations with specific characteristics and are often collected using cohort studies [1]. Patient registries have a clinical or public health-related purpose [1]. Registries have evolved from patient notes to electronic databases that can include large amounts of clinical and demographic data, as well as information on biological samples stored in biobanks [1]. An important feature of patient registries is the standardized and continuous prospective data collection done in a real-world setting, with patient management determined by the healthcare provider rather than by a study protocol [21]. Patient registries enrol a larger and more diverse patient population than controlled clinical trials, and can be a source of recruitment of patients for controlled clinical trials [21].

Patient registries can either be disease-specific or treatment-specific, and hospital-based or population-based [1]. An example of a population-based registry is the Public Health England (PHE) National Cancer Registration and Analysis Service (NCRAS) [22]. An example of a hospital-based registry program in the US is the American Academy of Orthopaedic Surgeons (AAOS) registry, which collects and analyzes national patient data and aims to improve orthopedic care and outcomes [23].

Real-world Data from Clinical Studies

Clinical studies that are not part of a controlled clinical trial and are conducted in a real-world setting can provide data for real-world evidence that is disease-specific or treatment-specific, and hospital-based or population-based [1,24].

Case-Control Studies

These are retrospective studies that can provide real-world data by identifying outcomes and the cause/s of a disease, either for rare diseases or diseases with a long duration between cause and disease occurrence [24]. Case-control studies are also valuable for the assessment of multiple variables to identify potential predictors of treatment outcomes [1].

Cohort Studies

Cohort studies assess the incidence, natural history, causes and risk factors, prognosis, and treatment outcomes, and can be retrospective or prospective [24].

Cross-Sectional Studies

Cross-sectional studies aim to assess the prevalence and outcomes of disease in a single group of patients, and have a role in obtaining real-world data on the prevalence of underdosing or achieving appropriate dose selection in patients treated in real-world situations [24].

Real-World Data from Pragmatic Clinical Trials

In 1967, Schwartz and Lellouch were the first to apply the word “pragmatic” as a descriptor for clinical trials designed to provide data to choose between care options, in contrast to an explanatory clinical trial used to test causal associations [25]. Currently, pragmatic clinical trials provide data to support evidence-based decisions for the adoption of the drug or medical product into clinical practice [26,27]. Pragmatic clinical trials can be prospective or retrospective and can provide results that can be generalized in real-world settings [28].

Real-World Data from Social Media and Patient Networks

Patients use social media to find health information, connect with and share experiences with other patients, and find health and social support, which may result in a large amount of potential real-world data [1]. Social media sites can also capture information on patient experiences, including adverse events, outcomes, patient perspectives, reasons for treatment non-adherence, and quality of life [1]. Currently, social media shows promise for real-world data acquisition for post-marketing drug safety surveillance [29]. However, it is less clear how this form of real-world data collection can be used for drug development, including pre-approval, to guide clinical trial development and compare similar drugs, and post-approval to evaluate off-label use [29]

Patient-powered research networks (PPRNs) are online platforms established and managed by patients and patient support groups and by research groups to collect data focused on specific diseases to compare the effectiveness of research and the use of patient-centered outcomes [1]. As an example, the National Patient-Centered Clinical Research Network, PCORnet, prospectively curates observational real-world data from EHRs [30]. PCORnet is funded by the Patient-Centered Outcomes Research Institute (PCORI) and includes eight Clinical Research Networks and 79 health system sites [30]. In 2019, PCORnet began linking real-world data sources to support studies and incorporate patient-centered economic outcomes research with links to commercial health plan partnerships, health billing systems, and pharmacy billing data [30]. Therefore, social media and patient networks have potential as a source of real-world data to guide drug development and drug pre-approvals [29]. However, guidance is required to encourage its use, and both regulatory and technological developments are required [29].

Real-World Studies: Some Lessons Learned

In situations where drug approvals are followed by rapid uptake and demand in the real world, drug side effects and interactions may be identified that were not reported during the conduct of controlled clinical trials that supported drug approval. A recent example comes from glucagon-like peptide 1 (GLP-1) receptor agonists for weight loss, as real-world evaluations that have now gone beyond the trial duration of 68 weeks [31]. A recent study by Thomsen and colleagues evaluated current real-world evidence on the use, effectiveness, and adverse events associated with new GLP-1 receptor agonists, liraglutide, semaglutide, and tirzepatide, compared with findings from controlled clinical trials [32]. A real-world evaluation showed that during the first year of GLP-1 receptor agonist use, weight reduction in clinical practice was lower, and drug discontinuation rates were higher (up to 50%), but with individuals using lower drug doses than in clinical trials [32]. Therefore, initial real-world analysis has highlighted some cautions in the use of GLP-1 receptor agonists and the remaining gaps in understanding early discontinuation, suboptimal drug dosing, and cost-effectiveness of GLP-1 receptor agonist use in real-world settings [32].

Real-world data have also supported treatments, medical devices, and diagnostic techniques as they undergo regulatory evaluation. A recent example is the diagnostic detection of circulating DNA (ctDNA) in liquid biopsy material. Detection of ctDNA has shown promise in cancer screening, and clinical trials have begun to evaluate ctDNA monitoring for cancer treatment response in non-small cell lung cancer (NSCLC), breast cancer, and colorectal cancer (CRC) [33]. A real-world study evaluated the impact of monitoring ctDNA on the detection and management of recurrence in 108 patients with resected early-stage NSCLC, of which 12 (11.1%) were ctDNA-positive, and 8/12 (66.7%) had a postoperative recurrence [34]. This real-world prospective study identified improved patient risk stratification using ctDNA monitoring, supporting further studies on the use of ctDNA in patient risk stratification [34].

The importance of standardizing and regulating the conduct and reporting of real-world studies is highlighted by the increasing number of registered studies. In 2023, Li and colleagues identified and reviewed real-world studies registered at ClinicalTrials.gov up to February 28, 2023 [35]. This retrospective analysis identified 944 registered studies from 48 countries, including China (37.9%), followed by the US (19.7%). Single-center interventional studies were most common (63%) and involved drug treatments (42.4%) and medical devices (9.1%), with 49.4% of studies having a patient participant number of 500 or more [35]. One-third of registered real-world studies (32.7%) involved topics on malignancy [35]. The authors also identified deficiencies in the description of the study design in the majority of registered real-world studies and in the study registration data [35].

Guidelines for Real-World Studies

Good clinical practice guidelines for the design, conduct, analysis, and reporting of observational real-world studies have been developed by the US FDA, the European Medicines Agency (EMA), the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP), and the European Network for Health Technology Assessment (EUnetHTA) [1,36–38]. These guidelines aim to improve real-world data from studies that adequately address methodological issues, including confounders, study bias, and unknowns that could affect decisions from regulatory authorities and healthcare providers [1].

In 2017, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) and the International Society for Pharmacoepidemiology (ISPE) formed a task force to develop consensus recommendations for good procedural practices in obtaining evidence-derived real-world clinical studies [39]. The ISPOR/ISPE Task Force recommendations included study registration, replicability, and stakeholder involvement in real-world studies [39]. These recommendations focused on good procedural practices for studies that involved testing a specific hypothesis in a particular population [39]. Two categories of studies were identified to provide real-world data on exploratory treatment effectiveness studies and hypothesis-evaluating treatment effectiveness (HETE) studies [39]. HETE studies aim to test a hypothesis in a specific patient population [39]. Both exploratory and HETE studies can provide insights based on clinical observations [39].

In June 2022, the UK National Institute for Health and Care Excellence (NICE) published a framework for real-world evidence, supported by a 2021 to 2026 national strategy, to use real-world data to resolve gaps in knowledge and to drive health innovations and access to them for patients [40]. The NICE framework describes best practices for planning, conducting, and reporting real-world evidence studies to generate evidence in a transparent way using analytical methods to minimize the risk of bias and identify knowledge gaps and areas of uncertainty [40]. NICE intends to regularly update the real-world evidence framework to reflect user feedback, clinical input, case studies, developments in real-world evidence methods, and priority healthcare topics [40]. The European Medicines Agency (EMA) and the European Medicines Regulatory Network (EMRN) have combined to form the Data Analysis and Real-World Interrogation Network (DARWIN EU) to coordinate timely and reliable evidence on the uses, effectiveness, and safety of medicines and vaccines from real-world healthcare databases in the European Union (EU) [41]. DARWIN EU aims to inform the EMA and national authorities in the EMRN throughout the lifecycle of a medicinal product with validated real-world data [41].

In 2021, Wang and colleagues proposed the development of STaRT-RWE, a structured template for planning and reporting real-world evidence studies, and to facilitate reproducibility, validity, and synthesis of real-world evidence [42]. In 2023, the ISPOR/ISPE Task Force created a harmonized protocol template for real-world evidence studies to evaluate treatment effects to inform clinical decisions [43]. The 2023 HARmonized Protocol Template to Enhance Reproducibility (HARPER) is designed with standard text and a tabular visual structure [43,44]. HARPER provides structure for the research process from study protocol to registration, study implementation, and reporting [43]. HARPER also incorporates recent insights on data and study details and transparency to enable reproducibility of real-world evidence studies [43]. The main goals of the protocol and template are to help investigators clarify the study rationale, to facilitate decision-making by making study biases clear, and to facilitate study reproducibility [43]. The results of the uptake and implementation of the 2023 ISPOR/ISPE HARPER template are awaited [43].

Future Directions

Real-world evidence has become integral to the evaluation of safety, efficacy, and implementation of drugs and medical devices, and sources of real-world data have been established and tested. However, there remain challenges due to the diversity and varied quality of the sources of real-world data [45,46]. Future developments and refinements in data acquisition and quality will include the use of artificial intelligence (AI) or machine learning methods to analyse medical text from sources such as EHRs and social media sites, with advanced analytics for unstructured data health technologies, including wearables [45,46]. Computational frameworks and algorithms are expected to become increasingly available to improve study design and identify patients who will benefit from treatment [45,46]. Initiatives have already begun to apply structured methods to design real-world evidence studies and analyse real-world data in the absence of controlled clinical trials [47,48]. Future partnerships between data analytics experts, companies, regulatory bodies, clinicians, and researchers will be required to develop more rapid and low-cost real-world evidence capabilities, monitor public health benefits, and facilitate improved databases and access to them.

Conclusions

Real-world evidence results from analysis of data from observational patient registries or retrospective or prospective observational studies and provides insights beyond data from randomized controlled trials to improve clinical decision-making. Real-world evidence and real-world data are vital to the safe and effective delivery of medical products to complement, support, and strengthen the findings and interpretation of data from controlled clinical trials. New analytical developments will support ongoing guidelines for data acquisition and analysis. However, patient access to healthcare guidance and clinically safe, effective, and affordable medicines requires continued awareness from all healthcare stakeholders.