Real‐World Evidence in Oncology: Opportunities and Limitations

Short abstract

The analysis of real‐world evidence to answer clinical and policy‐relevant questions in the field of oncology has gained increased interest in recent years. This article highlights the strengths and weaknesses of the use of real‐world evidence in cancer care.

Traditionally, randomized controlled clinical trials (RCTs) have been considered the highest level of evidence to define the efficacy of treatments, before their adoption in clinical practice. However, in oncology, like in other fields of medicine, the analysis of real‐world evidence (RWE) to answer clinical and policy‐relevant questions that cannot be directly or completely answered using data from RCTs has rapidly gained increased interest in recent years 1, 2, 3. It is clear that data obtained from health records, cancer registries, and other RWE sources can produce valuable insights into treatments and their outcomes in routine, daily oncology practice. However, caution must be paid to the intrinsic limitations of such data, to avoid a misleading and potentially harmful use of them.

Among several applications for RWE proposed in recent years, it is useful to consider the following: (a) RWE following the conduction of RCTs, to better define the effectiveness of treatments in clinical practice (including subgroups of patients excluded or under‐represented in RCTs); (b) RWE to describe reliability and transferability of complex procedures, such as molecular diagnostics and multimodal treatments in clinical practice; (c) RWE to define the effectiveness of interventions in settings where RCT are (still) not available, typically the case of rare molecular subgroups where the conduct of “traditional” RCTs can be particularly challenging; and (d) RWE to better define safety of treatment, especially in terms of long‐term adverse events.

Of course, whereas data coming from clinical trials are prospectively collected and verified with well‐established rules and procedures, which should guarantee acceptable quality of data, the collection of real‐world data poses several methodological problems, for instance, in terms of both data sources and data verification. Consequently, taking into account the opportunities and the potential limitations, our aim is to highlight some strengths and weaknesses of RWE in oncology.

Real‐World Evidence to Describe the Effectiveness of Treatments

Patients enrolled in RCTs are accurately selected, and on average, they may significantly differ if compared with the heterogeneous population of patients that physicians will evaluate and treat in daily clinical practice 4, 5. This is due to stringent eligibility criteria, such as good performance status and absence of clinically relevant concomitant diseases. For instance, the analysis of the Investigational New Drugs applications submitted in 2015 to the U.S. Food and Drug Administration, for oncology and hematology products, showed that 60% of the trials required Eastern Cooperative Oncology Group performance status 0 or 1 (practically excluding all significantly symptomatic and unfit patients), 77% excluded known active or symptomatic metastases to central nervous system (sometimes allowing the inclusion of patients with treated or stable brain metastases), 84% excluded patients with known or active human immunodeficiency virus infection, and 74% excluded patients with current—or history of—cardiovascular disease or risk (including angina pectoris, uncontrolled hypertension, myocardial infarction, congestive heart failure, and arrhythmia) 6. This implies that the “performance” of the experimental treatment in the population of subjects enrolled in the RCT could be significantly “diluted” in the daily clinical practice, because of lower compliance, reduced tolerability, increased competing risks of death, or worsening of clinical conditions. In other words, the “effectiveness” of a treatment (defined as its ability to do more good than harm when provided under usual circumstances of health care practice) could be substantially lower than its efficacy (defined as the extent to which an intervention does more good than harm under ideal circumstances) 7. Moreover, RCTs conducted in the adjuvant setting usually include patients at higher risk of relapse in order to increase the number of events and the probability to see an effect if it does exist. When the magnitude of efficacy shown in the clinical trial is substantially large, its reduction in daily life should not compromise the reproducibility of the trial results in clinical practice. However, a large proportion of new treatments only show a globally modest efficacy within RCTs, and their effect in clinical practice might be further diluted, letting their real value fall under an acceptable threshold of relevance; in this case, analysis of RWE can be useful to define effectiveness of those treatments.

Based on these considerations, some authors have suggested that regulatory agencies could take into account the restrictions in eligibility criteria of RCTs to “weight” their results: as trials become more unrepresentative of the real‐world population, regulatory agency could demand larger magnitudes of benefit before approval, in order to ensure that benefits will not be extremely diluted or completely lost in the subsequent application into the real world 8. In this scenario, the role of RWE could become even more important, given that the conduct of postmarketing studies could be of great value in confirming or refuting the drugs’ benefit on survival in real‐world populations.

Several examples of RWE analyses that challenge the magnitude of the efficacy previously shown in RCTs can be found in recent literature 9, 10. In patients with advanced hepatocellular carcinoma, sorafenib produced a significant improvement in overall survival (2–3 months prolongation in median survival) compared with placebo, within randomized trials that included only patients with well‐preserved liver function 11, 12. A subsequent analysis of patients treated in clinical practice suggested that the survival of patients treated with sorafenib—obviously less selected compared with the previous RCTs—was much shorter, questioning the reproducibility of the advantage compared with those patients who received no active treatment 9. Similarly, a comparison between patients with castration‐resistant prostate cancer receiving docetaxel plus prednisone within a clinical trial and those who received the same treatment off‐trial showed a significantly worse outcome in the latter group (in terms of both reduced survival and increased toxicity), supporting the hypothesis that results of RCTs establishing the efficacy of a new cancer treatment could translate into poorer outcomes and greater toxicity when applied in routine clinical practice 10. On the other hand, there is also an increasing number of examples of RWE analyses that, according to authors’ interpretation, have confirmed the efficacy demonstrated in RCTs (Table 1) 13, 14, 15, 16.

Table 1.

Selected examples of real‐world evidence, discordant or concordant with previous randomized controlled trials conducted in the same setting

Author [reference]	Setting	Patients	Sample size, n	Treatment	Outcome	Authors’ conclusions
Sanoff HK, 2016 9	Advanced HCC	Patients with advanced HCC diagnosed from 2008 to 2011 were identified from the Surveillance, Epidemiology, and End Results‐Medicare database.	422	Sorafenib	Median OS was 3 months. Difference in OS with a propensity score‐matched cohort of untreated patients was not significant.	“Survival after sorafenib initiation in newly diagnosed Medicare beneficiaries with HCC is exceptionally short, suggesting trial results are not generalizable to all HCC patients.”
Templeton AJ, 2013 10	mCRPC	Patients with mCRPC treated with docetaxel at Princess Margaret Cancer Centre, in routine practice or in clinical trials.	357 (314 routine practice, 43 clinical trials)	Docetaxel	Median OS was 13.6 months in routine practice and 20.4 months within clinical trials. Rates of febrile neutropenia were 9.6% vs. 0%.	“Survival of patients with mCRPC treated with docetaxel in routine practice is shorter than for men included in trials and is associated with more toxicity.”
Boegemann M, 2019 13	mCRPC	Patients with chemotherapy‐naïve mCRPC, treated in four European countries	481	Abiraterone acetate plus prednisone	Median TTF was 10.0 months; median PFS was 10.8 months.	“Treatment effectiveness in the real‐world is maintained despite patients having poorer clinical features at initiation than those observed in the pivotal trial population.”
Blomstrand H, 2019 14	Advanced pancreatic cancer	First 75 consecutive patients in the southeastern region of Sweden who received first‐line treatment with gemcitabine + nab‐paclitaxel.	75	Gemcitabine + nab‐paclitaxel	Patients with metastatic disease: median OS 9.4 months; median PFS 4.5 months. Patients with locally advanced disease: median OS 17.1 months; median PFS 6.8 months.	“This study confirms the effectiveness and safety of first‐line gemcitabine + nab‐paclitaxel in both locally advanced and metastatic pancreatic cancer in a real world setting.”
Lee JY, 2019 15	Platinum‐resistant advanced ovarian cancer	Patients with platinum‐resistant ovarian cancer treated in 27 Korean institutions.	391	Bevacizumab with single‐agent chemotherapy (weekly paclitaxel, pegylated liposomal doxorubicin, topotecan)	Median PFS 6.1 months	“In Korean ovarian cancer patients, the safety and effectiveness of chemotherapy with bevacizumab in a real‐world setting was consistent with the results from a randomized controlled study.”
Pasello G, 2019 16	Nonsquamous EGFR‐mutant advanced NSCLC	Patients with EGFR mutation receiving first‐line treatment within the Veneto Oncology Network (Italy)	109	Erlotinib or gefitinib or afatinib	Median TTF 15.3 months, without significant difference among the three drugs	“Real‐world data collection reporting a multicenter adherence and compliance to diagnostic‐therapeutic pathways defined for patients with EGFR‐mutant NSCLC.”

Abbreviations: EGFR, epidermal growth factor receptor; HCC, hepatocellular carcinoma; mCRPC, metastatic castration‐resistant prostate cancer; NSCLC, non‐small cell lung cancer; OS, overall survival; PFS, progression‐free survival; TTF, time to treatment failure.

After the registration of a new drug, RWE can be useful to describe the outcome of patients who were under‐represented in—or completely excluded from—pivotal trials. However, in this kind of study, the absence of randomization leads to inaccuracy in the estimation of the added benefit associated with treatment. Consequently, if the major issue is about treatment tolerability in a specific group of patients (as is often the case with older subjects), the absence of a control group might be considered a minor problem, given that the main objective of the analysis is the description of adverse events. On the contrary, if the major issue is about the dilution of efficacy due to worse prognosis and concomitant diseases, the absence of a control group is a major limitation of the study. In the absence of randomized evidence, the treatment of these subgroups of patients cannot be defined 17.

Similarly, a control group is absolutely needed if the objective of RWE is to define the effectiveness of treatment in settings where RCTs have not been performed. Comparative effectiveness research, defined as the conduction of observational studies that compare the outcomes of two or more treatments in a real‐world population of patients not randomly assigned, has been proposed as a method to analyze the relative outcome of different treatments 1. However, these studies are inherently limited by selection bias, given that treatment choice in routine clinical practice is strongly influenced by the baseline characteristics of each patient. Several statistical methods have been developed and applied to contrast this bias (e.g., multivariate regression analysis, propensity score analysis), but these methods do not eliminate the bias and should not be considered an alternative to the conduction of RCTs. The comparison of the clinical outcomes of nonrandomized groups of patients who have received different treatments in routine clinical practice remains methodologically weak and problematic. Consequently, comparative effectiveness studies need to be designed and interpreted with great caution.

RWE could produce useful data in terms of treatment sequence, considering that one of the major limitations of the evidence produced by RCTs is that most trials are focused on the comparison of treatments within a specific line of therapy and are not designed to allow comparisons of sequences. Patients treated in a second‐line trial could have not necessarily received the current first‐line standard treatment, and patients treated in a first‐line trial could have not necessarily received, after disease progression, the currently available second‐line standard treatment. From this point of view, RWE could integrate the evidence of RCTs, especially in those treatment settings characterized by the recent introduction of therapeutic news. For instance, in the setting of human epidermal growth receptor 2 (HER2)‐positive breast cancer, the population of patients of the pivotal trial of second‐line trastuzumab emtansine (T‐DM1), showing the efficacy of the drug and leading to its approval for clinical practice, had not received pertuzumab as part of their first‐line treatment 18. RWE can be helpful to produce data about these sequences 19.

Results of pivotal RCTs are expected to be applied all over the world, but patients in different countries can be substantially different in terms of ethnicity, characteristics of disease, and treatment algorithms. Furthermore, the application of the results of an RCT could have substantially different economic implications within different countries and within different health systems. From this point of view, however, RWE represents a unique opportunity of studying and discussing the application of a treatment within a specific geographic and economic context 16. Within each specific reality, policy‐makers could develop mechanisms for re‐reviewing real‐world data on effectiveness and cost‐effectiveness data for all cancer drugs, in order to ensure fair and equitable access to cancer drugs, or at least to optimize the allocation of resources 20.

Table 2 lists the main strengths and weaknesses of efficacy description with RWE. In synthesis, RWE represents a relevant opportunity to define the effectiveness of treatments in clinical practice. However, because of their inherent methodological limitations, RWE studies should not be used as substitutes for clinical trials. RCTs and RWE should remain complementary forms of medical evidence 2.

Table 2.

Strengths and limitations of real‐world evidence about treatment efficacy

	Strengths of RWE	Limitations of RWE
Description of treatment efficacy in “clinical practice” heterogeneous population	Lower selection bias in study population compared with RCTs	The absence of a control group does not allow the accurate estimation of the efficacy compared with previous/alternative standard Quality of data sources, collection, and verification could be lower compared with clinical trials
Description of treatment efficacy in special patient populations	Potential focus on efficacy in special patient populations, often under‐represented in (or excluded from) RCTs
Evaluation of efficacy in settings where an RCT has not been performed (e.g., rare subpopulations)	Production of evidence in a setting suffering from the absence of an RCT
Nonrandomized comparison of patients receiving different treatments for the same condition	Production of evidence in a setting suffering from the absence of a direct comparison	Selection bias is inherent in nonrandomized groups and cannot be avoided even with statistical techniques (e.g., propensity score, multivariate analysis) Quality of data sources, collection, and verification could be lower compared with clinical trials
Use of treatments within specific geographic and/or economic contexts	Production of evidence in patients with different characteristics compared with RCTs (due to ethnicity, characteristics of disease, other treatments) Production of pharmaco‐economic data within a specific country or a specific health system	Results produced within a specific geographic and/or economic context cannot be applied to different contexts

Abbreviations: RCT, randomized controlled trial; RWE, real‐world evidence.

Real‐World Evidence for Rare Molecular Subpopulations

Traditionally, the authorization for use in clinical practice of new anticancer drugs follows a clinical research program, in which the demonstration of promising activity in a phase II trial is followed by a randomized phase III trial that compares the experimental treatment with the best treatment already available for that specific condition. This model has been applied, in recent years, to study drug efficacy in a relatively rare population, as is the case with molecularly selected subpopulations of specific types of cancer.

However, molecular characteristics of tumors have revealed many molecular alterations that are present in a very small proportion of cases. In principle, the conduct of traditional clinical trials in these molecular subpopulations is feasible, but it implies the screening of a very high number of subjects to find potentially eligible cases. This could represent a major obstacle to the development of treatment options for these patients: the cost (and the risks) of a clinical trial could likely be too high to attract the interest of pharma companies.

Could RWE help provide evidence in this particular setting? In principle, the use of a drug in the real‐world setting should follow the demonstration of efficacy within a clinical research program. But what if the latter does not exist? Could an RWE be produced even in the absence of clinical trials? As a matter of fact, the availability of tests for broad molecular characterization of the tumor tissue is rapidly increasing, and as a consequence, the number of cases with known molecular alterations targeted by drugs that are available in clinical practice, but registered for different tumors, is rising as well. For instance, predictive role of HER2 increase in gene copy number and overexpression for the use of anti‐HER2 drugs is well established in breast cancer 21 and in gastric cancer 22, and some evidence has also been produced in colorectal cancer 23. What if the same molecular alteration is found in another type of solid tumor? The presence of the molecular alteration itself is not a sufficient condition to ensure the efficacy of a targeted treatment, even if the same treatment has proved effective in other types of tumors carrying the same alteration. When the BRAF targeted drug vemurafenib, initially developed in BRAF mutated melanoma, has been tested within a basket trial in different tumors carrying the same BRAF V600E mutation, the administration of the drug was associated with high activity in some types of tumors (e.g., lung cancer) but with a virtual lack of activity in other tumors unless combined with other targeted agents inhibiting compensatory loops (e.g., colorectal cancer) 24. This means that the use of targeted agents in clinical practice, without any previous demonstration of efficacy, should be discouraged, because it could represent a toxic, ineffective approach as well as a waste of money.

The problem is that sporadic, uncontrolled, real‐world, off‐label use not based on scientific evidence, at times leading to the publication of positive case reports, creates a very high risk of publication bias. More than a real‐world, uncontrolled use of targeted agents for “orphan” conditions, a prospectively defined experimental program could be better associated with production of useful evidence. Some years ago, this model was proposed by Richard Schilsky, discussing the potential scenarios of precision cancer medicine 25. Starting from the problem of patients with advanced cancer, with a potentially actionable gene alteration detected by molecular profiling, he emphasized the opportunity of linking clinical research and clinical care, gaining valuable insights from a massive number of patients. In fact, these cases present two distinct problems: (a) how to get the drug and (b) how to learn and obtain useful evidence from the treatment. In the absence of recruiting clinical trials, and in the absence of expanded access programs (or compassionate use or similar), the only opportunity to use the drug could be an off‐label use, which poses serious problems of reimbursement and out‐of‐pocket cost. However, the opportunity to create what Schilsky defined as a “facilitated drug access program”: a “formulary” of targeted drugs (already marketed for different tumors), followed by the creation of a prospective registry of patients and their outcomes. This would allow to expand indications for a targeted agent beyond already approved indications. Of course, such a program implies the agreement of all stakeholders: pharma companies should agree to provide the drug for free within the program, physicians should agree to administer the treatment and collect data including patients’ outcomes, and patients should agree to allow collection of their data. The program should be supervised by an “honest broker,” namely, a scientific society, who guarantees the quality of data collection. At least in theory, regulatory agencies could use these clinical data for subsequent decisions about the use of the drug in that specific new indication. Such a program would have several benefits for all parties involved: (a) patients become “cancer information donors” with the opportunity to receive an otherwise unavailable targeted treatment, matched to their molecular profile; (b) physicians would benefit from the participation in a program allowing the offer of new opportunities to patients; (c) pharmaceutical companies would receive information about activity of their drugs in settings not yet explored in the “official” clinical research program; and (d) payers like national health system or insurance companies would not have any additional cost for the experimental drug, potentially covering these costs in the future, following a successful real‐world trial. Of course, considering that the eligibility of patients within such a program is necessarily prospectively defined and that data about treatment and outcome need to be carefully collected, these kinds of projects, more than being a “typical” real world evidence, are de facto more similar to the conduction of an academic trial. Based on this experimental approach, prospective projects are ongoing, that are explicitly planned, designed, and conducted as clinical trials 26.

Real‐World Evidence and Description of Treatment Safety

An accurate description of the toxicity profile of anticancer treatments has crucial clinical implications because it will inform physicians and patients regarding the safety of each therapy, giving important information about what to expect—and what to communicate—when starting that treatment in a new patient in everyday clinical practice. After authorization for clinical use and introduction of new treatments in clinical practice, real‐world data can play a substantial role in the optimal definition and characterization of tolerability and adverse events associated with administration of anticancer treatments. These data could also be useful for establishing a definitive analysis of benefits and risks associated with treatment for clinical practice guidelines: in recent years, safety analyses based on RWE have been increasingly cited in the National Comprehensive Cancer Network and the American Cancer Society practice guidelines 27.

By the time of treatment licensing in clinical practice, the limited number of patients exposed within clinical trials does only allow the detection of the more common adverse drug reactions, but rare toxicities might have been missed. Even when observed, the small number of events could make an accurate estimate of the absolute and relative risk associated with the experimental drug difficult. From this point of view, postmarketing reporting of adverse events is crucial. Among recent examples of toxicities that were probably too rare to be appreciated and emphasized within the pivotal trials, and have been better characterized postmarketing, there is cardiac toxicity of immune checkpoint inhibitors 28. In addition, postmarketing surveillance and production of real‐world data are important to obtain information about chronic toxicity, toxicity profile in special patients population (such as older subjects), or drug interactions that are often incomplete or not available.

As a matter of fact, despite the quality of data collection ensured by good clinical practice rules, the estimation of the risk of adverse events based on the information collected within pivotal trials can be suboptimal, owing to the limited number of patients included in the trial, resulting from the restrictive eligibility criteria, and also as a result of the limited follow‐up duration at the time of data analysis and publication. Furthermore, description of adverse events in publications of clinical trials can be suboptimal, particularly in the reporting of recurrent (i.e., those events occurring more than once in the same patient) or late toxicities and in the description of toxicity duration 29. In a systematic review of 81 trials of targeted therapies and immunotherapies approved by the U.S. Food and Drug Administration between 2000 and 2015 for solid malignancies in adult patients, more than 90% of trials scored poorly in their reporting of recurrent and late toxicities and in reporting the duration of adverse events.

Long‐term safety is a relevant information especially for patients receiving the treatment in a potentially curative setting (e.g., adjuvant), but even in the advanced stages of disease, when treatment is associated with a chance of durable benefit, at least in a minority of patients, as we are experiencing, in several solid tumors, with the use of immune checkpoint inhibitors.

Of course, data quality can be an issue in the description of adverse events in the “real world.” As a general rule, toxicity is best described when it is prospectively and explicitly studied (and ideally with the administration collection of patient‐reported outcomes related to subjective symptoms). Accordingly, postregistration studies specifically designed to collect information on tolerability are a precious instrument to complement the information derived from RCTs. On the other hand, retrospective RWE analyses such as collection of information from health records and other data sources could be subject to underestimation of adverse events, considering that under‐reporting of subjective symptoms in medical health records is a common phenomenon. Table 3 lists the main strengths and weaknesses of safety description with RWE.

Table 3.

Strengths and limitations of real‐world evidence about treatment tolerability and adverse events

	Strengths of RWE	Limitations of RWE
Description of rare toxicities	Potentially larger number of patients compared with those enrolled in RCTs	Lack of experience of doctors in clinical practice in appreciating rare drug‐related toxicity
Description of tolerability in “clinical practice” heterogeneous population	Lower selection bias in study population compared with RCTs.	Description of adverse events could be less accurate compared with the close monitoring and prospective collection within RCTs
Description of tolerability in special patient populations	Potential focus on tolerability in special patient populations, often under‐represented in (or excluded from) RCTs
Description of long‐term toxicities	Longer follow‐up compared with primary analysis of RCTs: useful for description of long‐term toxicities
Accuracy of results	If designed as prospective collection of safety data in the “real‐world” setting, good accuracy of results	If designed as retrospective collection of safety data in the “real‐world” setting, lower accuracy of results and risk of “falsely reassuring” data. Establishment of a causal relationship between the drug and the adverse events can be more difficult compared with prospective clinical trials.
Incorporation of PROs into description of toxicity	If designed as prospective collection of safety data in the “real‐world” setting, possibility of inclusion of PROs in the description of AEs	If designed as retrospective collection of safety data in the “real‐world” setting, no possibility of inclusion of patient‐reported outcomes in the description of AEs

Abbreviations: AEs, adverse events; PROs, patient‐reported outcomes; RCT, randomized controlled trial; RWE, real‐world evidence.

Real‐world studies can also give useful information about the compliance or adherence to treatment, that can be conditioned by toxicity but also by complexity of the regimen, high medication costs (at least in some countries), patient's age, and poor communication between the physician and the patient. The description of drug adherence in a real‐life setting is a growing concern to stakeholders such as payors and health care systems, considering that relevant divergence from the results of pivotal clinical trials in terms of tolerability and adherence could significantly expand the gap between treatment efficacy and effectiveness. Drug adherence can also be suboptimal in RW because of less motivated patients and physicians less trained to deal with toxicities.

Conclusion

Real‐world evidence can be useful to describe treatment efficacy in a “clinical practice” population characterized by lower selection bias compared with RCTs, adding details on the outcome of special patients’ populations, under‐represented or excluded from pivotal clinical trials. In addition, RWE could produce data about effectiveness and cost‐effectiveness in different geographic and/or economic contexts. Of course, the absence of a randomized group does not allow to estimate the efficacy compared with other treatments, and quality of data could be lower compared with clinical trials.

As for the evaluation of targeted treatments in rare, molecularly selected subpopulations of patients, prospective projects—that are explicitly planned, designed, and conducted as academic clinical trials—will likely produce more solid and methodologically sound evidence compared with the uncontrolled report of “real world” sporadic, off‐label use.

Although the description of adverse events in the real‐world practice could be less accurate compared with the close monitoring and prospective collection within RCTs, RWE can be useful for a better description of treatment tolerability, collecting adverse events in a larger number of subjects and within a more heterogeneous population compared with clinical trials.

Disclosures

Massimo Di Maio: Merck Sharp & Dohme, Takeda, Pfizer, AstraZeneca, Janssen (C/A), Bristol‐Myers Squibb (H), Tesaro (RF); Pierfranco Conte: Eli Lilly and Company, Novartis, Roche, AstraZeneca (C/A). Francesco Perrone indicated no financial relationships.

(C/A) Consulting/advisory relationship; (RF) Research funding; (E) Employment; (ET) Expert testimony; (H) Honoraria received; (OI) Ownership interests; (IP) Intellectual property rights/inventor/patent holder; (SAB) Scientific advisory board