Generating fit-for-purpose real-world evidence in Asia: How far are we from closing the gaps?

Abstract

Evidence generated by randomized controlled trials (RCTs) does not often represent the patient journey and clinical outcomes in the real world due to limited external validity or generalizability. Studies based on real-world data are intended to generalize results to the broader population; however, if the influence of external factors or confounders is not effectively managed, the cause-and-effect relationship and internal validity may be challenged, resulting in flawed results. The collection of quality real-world evidence (RWE) is crucial in Asia as there is often an underrepresentation of Asian populations in RCTs. In addition, few countries in Asia are catching up with the Western world in issuing practical foundational principles and guidance for conducting and adopting evidence for regulatory and reimbursement decisions. However, privacy and data protection laws are generally lagging behind technological developments in electronic medical records. While leveraging RWE in clinical and regulatory decision-making holds excellent potential, collective efforts across industry, governments, and research institutions are required for generating standardized practices and building capabilities for developing fit-for-purpose RWE in Asia.

INTRODUCTION

The terms real-world data (RWD) and real-world evidence (RWE) are often used interchangeably.[1,2] Still, these are interrelated concepts with more profound meaning and differentiation. RWD refers to the information collected during routine clinical practice. In contrast, RWE refers to well-designed studies based on RWD collected outside traditional clinical trial programs adopting randomized controlled trials (RCTs).[2,3] Table 1 lists a few regulator definitions for RWD and RWE.[4,5,6,7,8]

Table 1.

Regulator definitions for real-world data and real-world evidence

	RWD	RWE
FDA (United States)[4,5]	“Data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources”	“Clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD”
	FDA identifies three categories of potential study designs that may use RWD to generate RWE[6,7]
	Randomized designs using RWD (PCTs*)
	Nonrandomized, single-arm trials with external RWD controls; and
	Observational/noninterventional studies
EMA (Europe)[4,8]	“Healthcare related data that is collected outside of randomised clinical trials”	“Evidence coming from registries, electronic health records, and insurance data”

^*PCTs are prospective, randomized trials that aim to better reflect real-world clinical practice in terms of patient enrolment and trial conduct. Data can be collected (i) in a usual care setting, (ii) using "hybrid" approaches that combine existing data with primary data collection, or (iii) through EHR, claims, or registries. Patient follow-up is provided in a naturalistic setting. EHR=Electronic health record, EMA=European medicines agency, FDA=Food and Drug Administration, PCTs=Pragmatic clinical trials, RWD=Real-world data, RWE=Real-world evidence

RCTs are usually required to provide sound evidence regarding the efficacy and safety of a new medicine for its market authorization. A strong comparable foundation is possible between the study arms, provided all other influencing factors are equally balanced to draw the causality of target treatment on the outcome. Randomization allocates trial participants to either the experimental or controlled arm(s) with an aim to theoretically balance all of the other factors or confounders that may influence the cause-effect relationship.[9] However, individual subject variation and chance are still unavoidable. These factors introduce uncertainty into the trial, resulting in a large P value and wide confident intervals with an unreliable interpretation of results. Carefully designed eligibility criteria constrain variations across study participants and help to reduce the possible error(s) in study results caused by the accidental mismatch of confounders, even in the presence of randomization. This, however, yields a more homogeneous study population but imparts internal validity.[10] Internal validity refers to the ability to use a study result to draw correct inferences on the true causal effect of a treatment on the outcome. Internal validity enables confidence in the integrity of results, and a well-designed RCT often achieves good internal validity.[11]

However, the evidence generated by RCTs may not truly represent patient outcomes in the real world.[1,2,12] RCTs require the random allocation of carefully selected study participants who are treated and examined under strictly controlled conditions with regular but limited follow-up. This enhances the internal validity but might jeopardize the ability of the results from a study to be generalized to a broader yet relevant population.[13] External validity refers to the clinical study’s capability to generalize a study’s results to the population of interest.[11]

Though RCTs are considered the gold standard to generate evidence for the cause-and-effect of treatment, they are not always practical in the real world. The limited number of target patient population, low incidence of treatment endpoints, preference for other available treatments, highly diverse clinical settings, and lack of standard practice are potential challenges in initiating or completing RCTs. Early termination or inability to recruit sufficient patients is not uncommon with RCTs.[14,15] Furthermore, it is impossible to adopt an RCT study design to include and comprehend all diverse populations in different clinical settings.

In real-world clinical practice, physicians very often treat patients who do not exactly fulfill the eligibility criteria of RCTs, such as those with comorbidity, extreme age groups, concomitant treatments for the same or other medical conditions, and specific demographic characteristics with a low representative in large RCTs. Studies based on RWD include heterogeneous populations with diverse patient backgrounds under routine clinical practice.[16] RWE generated from well-conducted RWD studies helps to supplement the results from RCTs with external validity to wider patient populations. However, a lack of an effective strategy to manage the confounding effect is a well-known major threat to the internal validity of RWE. Confounding is the influence of external factors on the outcome, often distributed unevenly in study arms. Confounding factors distort the observed cause-and-effect relationship and affect the internal validity.[3] Several strategies, either through study design or data analysis, can increase the internal validity of RWE. These could include a prospective evaluation with adequate sample size and time to follow-up and adjusting the confounding effect through preidentification and matching the known confounders.[12,17] Overall, RCTs and RWE complement each other in completing the evidence for consideration of internal and external validity. Figure 1 depicts the key strengths and weaknesses of RCTs versus RWD studies.[1,2,12]

Figure 1.

Strengths and weaknesses of RCTs versus RWD studies. RWE complements RCTs by generalizing the results to a broader patient population. However, confounding due to lack of randomization often limits the potential of RWD. Confounding can be minimized either through careful study design or through data analysis. These could include a comparison or control group, restricting the selection of patients in the treatment group to those who are new users (of a particular treatment)/only to adherent patients and carrying out appropriate and adequate matching (to create comparable cohorts in the real world, after observing how a doctor decides which drug for which patient) and sensitivity analysis based on the observed confounders. RWD = Real-world data, RCTs = Randomized controlled trials

The increasing importance of RWE has also been underlined by the United States Food and Drug Administration’s (FDA’s) 21^st Century Cures Act (Cures Act), supporting the adoption of RWE trials as important sources of evidence to be used for the regulation of medicines and biologics.[18,19] Further guidance on the utility and adoption of RWE has been issued by other major regulators, such as European Medicines Agency, Pharmaceuticals and Medical Devices Agency (PMDA) for Japan, National Medical Products Administration (NMPA) for China, and Taiwan FDA (TFDA).[20,21] Regulatory authorities may also mandate RWD studies or postmarket commitment/surveillance studies as a condition of market authorization.[20,21] Collection of RWE through well-designed RWD studies is even more crucial in Asia. Only around 17% of clinical trials are conducted in Asia,[22] and there is often an under-representation of Asian populations in pivotal clinical trials.[22,23] In most Asian countries such as India, Indonesia, Malaysia, China, Philippines, Singapore, and Thailand, reimbursement decisions are not made at market entry. RWE provides certainty about the effectiveness of technologies in the local setting for these countries.[23] Several of these countries also have a weaker reimbursement system and are still grasping the principle of health economics.[16] In other Asian countries such as South Korea, Japan, and Taiwan, reimbursement decisions coincide with market entry timing shortly after regulatory approval, and RWE is considered when re-assessing initial funding decisions or for price adjustment.[23]

The health care, research, and pharmaceutical leaders are optimistic that RWE will be increasingly accepted for regulatory, reimbursement, and clinical decision-making in the future in Asia.[24] However, currently, RWE is mostly only used for postmarketing analysis and clinical decision-making in Asia. In addition, the lack of adequate infrastructure to support its generation and issues with trust in the reliability of RWE limit its scope and potential for regulatory and reimbursement decision-making in Asia.[24] This review discusses the opportunities, gaps, and prospects for fit-for-purpose RWE generation, with insights for Asia.

WHERE ARE THE GAPS IN GENERATING FIT-FOR-PURPOSE REAL-WORLD EVIDENCE IN ASIA?

Biopharmaceutical companies leverage RWE to inform internal decisions made throughout the product development process, guide pipeline and portfolio strategy, inform clinical development, etc.[25] In clinical practice, RWE sheds light on medicines’ effectiveness and usage in different health-care settings, long-term safety data, patient preferences, dose adjustments, adherence, etc.[26] Health economic experts leverage RWE studies to evaluate the pharmacoeconomic and overall cost savings in health care and society.[27,28] There have been quite a few successes for drug regulatory approvals in oncology and rare diseases based on single-arm trials using external control arms with robust and authentic RWE.[4] Apart from these drug approvals, RWE has also been foundational for specific label expansions approved by regulators [Table 2].

Table 2.

Examples of label expansions approved by regulators based on real-world evidence

Drug and indication (Health Authority, Year)	RWE use examples
Prevnar 13 (PCV13) for prevention of acute otitis media in children (Health Canada, 2019)[4]	Expansion of existing approved pediatric indication to include acute otitis media in children aged, from 6 weeks to 5 years, using RWE from the national ambulatory medical care survey and national hospital ambulatory medical care survey
Palbociclib (Ibrance) for HR+, HER2-advanced/metastatic breast cancer (FDA, 2019)[4]	Label expanded to include treatment in males based on postmarketing reports and electronic health records as part of the totality of evidence
Paliperidone Palmitate (Invega Sustenna) for the treatment of schizophrenia (FDA, 2018)[4,29]	PRIDE study: The first example of the use of RWE from a pragmatic trial in schizophrenia to support an expansion of the label with a pragmatic endpoint of “delaying time to treatment failure” in patients who had prior contact with the criminal justice system. Data were added to the previously approved FDA label

FDA=Food and Drug Administration, PCV=Pneumococcal conjugate vaccine, RWE=Real-world evidence, PRIDE=Paliperidone Palmitate Research in demonstrating effectiveness

In Asia, the NMPA in China and the TFDA in Taiwan have published RWE guidance documents for supporting drug development and evaluation in 2020.[20] In 2021, PMDA in Japan released basic principles on the use of registries in approval applications and points to consider for ensuring reliability when registry data are used for approval applications.[30,31] South Korea and Singapore regulators have also been expressing increasing interest in leveraging RWE beyond postmarketing drug surveillance. In South Korea, government organizations, industry and academia leaders, and patient groups have started discussions to develop draft guidance for integrating RWE with reimbursement and regulatory decisions.[32] The Health Science Authority in Singapore is also developing guidance on essential considerations for RWD in supporting regulatory decisions.[33]

Other countries in Asia, however, do not yet have any formal RWE regulatory frameworks and struggle with capabilities to conduct high-quality RWE studies.[20,23] An online survey of using RWD/RWE to inform Health technology assessment (HTA) for reimbursement decisions in 11 health systems in Asia (Bhutan, China, India, Indonesia, Japan, Malaysia, Philippines, Singapore, South Korea, Taiwan, and Thailand) was conducted in 2019 by the National University of Singapore and Health Intervention Technology Assessment Program.[23] Some respondents acknowledged that they might not have the capability to assess whether confounding is correctly accounted for when determining RWD/RWE for decision-making. Other key challenges encountered by HTA agencies in adopting RWE for decision-making in Asia include: (i) Potential selection bias limiting external validity due to insufficient evidence that the patients selected to generate the RWE truly reflect the patients in local routine clinical care; (ii) Inadequate reporting in HTA dossiers if the patients in the RWD studies received other treatments or had comorbidities; and, (iii) Lack of clinician, institutional, or legislative support for data collection.[23]

The correction for confounding factors is generally tricky due to the inherent nature of study design in RWD studies.[2,13] At the same time, RWD databases in many centers in Asia do not contain full data to capture adequate information regarding potential confounders. The lack of guidelines on the quality of databases for generating RWE, collating data across centers and missing regulatory framework further adds to the complexity.[2,3,22]

Other practical challenges are engraved in the fact that several countries in Asia are still without sufficient data protection mechanisms. Privacy and data protection laws are lagging behind technological developments in Asia.[34,35] Many countries have not yet issued specific regulations/policies directed against the privacy protection of citizens’ health data.[35,36] Most of the countries in South Asia are still in the process of developing data privacy laws with no visible progress toward producing region-relevant law(s).[35] Data from the health-care industry are regarded as being highly valuable, and the health-care industry is most susceptible to data pilfering.[37] An analysis of hacking/Information Technology incidents and unauthorized access incidents in a 15-year timeframe shows that the health-care sector has faced the highest number of data breaches.[37] As electronic health records become more commonplace in Asia, concerns regarding patient confidentiality, privacy, informed consent, and data security remain a major issue, especially in resource-poor contexts.[38]

BRIDGING THE GAPS IN STAKEHOLDER ACCEPTABILITY OF REAL-WORLD DATA STUDIES

Studies based on RWD are not limited to collecting evidence for correct causality of treatment effect.[39] The information on epidemiology is equally essential in making clinical and public health decisions. Epidemiology information could include the distribution of the disease, its pattern and course of progression, changing trends in its incidence and prevalence, associated treatment patterns, and burden in the target community.[40] Furthermore, randomization is irrelevant in generating and collecting data for this type of information. Hence, community-based RWD with good coverage and consistent quality is crucial for enhancing disease screening and diagnosis, and transforming patient care. The coronavirus disease of the 2019 (COVID-19) pandemic is a practical example of how RWE positioned itself as a valuable and powerful tool.[2] Real-word data collected worldwide allowed researchers to understand the disease quicker, gather information on how to manage it, and deepen learning.[2] The importance and the need for ongoing preparedness in virology, vaccine technology, public health infrastructure, and rapid real-time access to valid data have been demonstrated.[41] Access to RWE helped us fully understand the pandemic’s magnitude and effects in real time. Measurement of the impact of public health measures in combating the spread of COVID-19 has led to preventable mortality and adverse events and comorbidities.[41]

Transparency and reproducibility of RWD are, however, essential for generating fit-for-purpose RWE.[42] Transparency in what the investigators initially intended to do protects against data dredging and cherry-picking of results. It can be achieved with preregistration and public posting of protocols before the initiation of analysis. Transparency makes direct replication possible – the validity of the design and operational decisions can be evaluated, questioned, and improved. Reproducibility is a feature of a study or a finding. A reproducible study could be from “direct replication” or “conceptual replication.” Direct replication refers to independent investigators implementing the same methods for the same data and still obtaining the same results. Conceptual replication refers to a reproducible finding tested by conducting multiple studies that evaluate the same question but use different data and/or apply different methodologies. Defining a statistical analysis plan before starting data collection and analysis, using suitable databases to respond to the research question, study registration, and commitment to publishing and matching and adjusting for potential confounders as well as a sensitivity analysis to test the robustness of the study are the key mechanisms to enhance the transparency and reproducibility of RWE.[2]

There are current unmet needs for leveraging best practices within and outside Asia for capability building and collaborative partnerships in the health-care industry.[43,44] Standardization, collaboration, and synchronization across the health-care industry, HTA agencies, and regulatory bodies are required for guidance on the assessment and promotion of transparency and reproducibility of RWE in Asia. This requires the establishment and implementation of appropriate policies and guidelines on transparency, data protection, patient consent, quality assurance, approval of data collection based on upfront intended use, and data ownership. Medical journals also need to establish policies to promote stringent criteria for the assessment of credibility of RWE.[2,44] According to the General Data Protection Regulation in the European Union, the patient’s personal data must be relevant and adequate but limited to what is necessary to the reasons for which it is managed. “Data minimization” can be achieved through a process of pseudonymization which refers to the de-association of a subject’s identity from the personal data being processed.[45]

The REALISE (RWD In Asia for HTA in Reimbursement) working group is an excellent example of collaboration between global experts and leaders from the HTA agencies across Asia. The REALISE group has developed a nonbinding guidance framework to generate and use RWD and RWE consistently and efficiently for research and clinical decision-making in Asia.[22] However, further guidance from public and private institutions across countries in Asia aligned with the recommendations from decision-makers is required on how to select the most appropriate study design components to limit bias and how to confirm the method chosen was appropriate.[2,44,46]

CONCLUSION

The use of RWD and RWE holds excellent potential for all stakeholders in health care in Asia, as patients are often underrepresented in global registrational RCTs. However, the potential of RWE will only be realized if stakeholders share responsibility and adopt a collaborative approach to overcome these challenges in generating quality RWE. Research stakeholders in health care and pharmaceutical leaders, along with policymakers, will need to work collectively to develop standardized practices for generating and leveraging quality RWD fit for regulatory and drug reimbursement decision-making in Asia.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.